Unfit to lead

After Lincoln’s death many criticized his successor. He was seen as unfit to take over, both due to his actual abilities, and his prejudices.

“What we do know is that Lincoln was succeeded in office by a man who lacked all the qualities of greatness that he possessed… Andrew Johnson was self-absorbed, insensitive to the opinions of others, unwilling to compromise, and unalterably racist.” (334)
Johnson was quite racist, and especially during the era of reconstruction, this impeded his ability to lead or understand what was best for the country and its citizens. (180)
This is especially visible in his third annual message, which explicitly stated many deeply racist ideas.
“Yet what was most striking about the message was its blatant racism; what had been muted in the Freedmen’s Bureau veto now became explicit.” (250)
“The subjugation of the States to Negro domination would be worse than the military despotism under which they are now suffering. It was believed beforehand that the people would endure any amount of military oppression for any length of time rather than degrade themselves by subjection to the Negro race.” Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999
“It is not proposed merely that they shall govern themselves, but that they shall rule the white race, make and administer State laws, elect Presidents and members of Congress, and shape to a greater or less extent the future destiny of the whole country. Would such a trust and power be safe in such hands?” Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999
“On the contrary, wherever they have been left to their own devices they have shown a constant tendency to relapse into barbarism.”Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999
“The great difference between the two races in physical, mental, and moral characteristics will prevent an amalgamation or fusion of them together in one homogeneous mass. If the inferior obtains the ascendency over the other, it will govern with reference only to its own interests for it will recognize no common interest--and create such a tyranny as this continent has never yet witnessed.”Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999
“Already the Negroes are influenced by promises of confiscation and plunder. They are taught to regard as an enemy every white man who has any respect for the rights of his own race.” Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project 

Johnson was chosen for the position of Vice President in an attempt to project unity. As a Southern Democrat, Johnson was the perfect candidate for a nation torn apart by conflict. His presence represents Lincoln’s commitment to keeping the nation together, and reintegrating the South. That being said, he was not chosen for his ability to be the next President, especially during such a fraught and influential period. In making the choice of Johnson as his VP, Lincoln was taking a risk, one that did not pay off after his assassination. Johnson’s Reconstruction was very different than Lincoln’s would have been.

“His presence on the ticket symbolized the party’s determination to reward Southern Unionists and extend the Republican organization into the South.” (44)
“The Executive (my predecessor as well as myself) and the heads of all the Departments have uniformly acted upon the principle that the Union is not only undissolved, but indissoluble.”Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999
Johnson’s Reconstruction was very lax, and ended with the same people in power as had been during the Civil War. 
“It is clear to my apprehension that the States lately in rebellion are still members of the National Union. When did they cease to be so?”Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999
“The stark truth was that outside the Unionist mountains, Johnson’s policies had failed to create a new political leadership to replace the prewar “slaveocracy”--partly because the President himself had so quickly aligned with portions of the old elite.” (197)
“Indeed,  while Johnson always claimed that his Reconstruction policy merely continued Lincoln’s, in crucial respects it was very much his own. On the one hand, Lincoln, at the end of his life, had made clear that he favored a limited suffrage for Southern blacks; on the other, he had never suggested exemptions to Presidential amnesty as sweeping as those contained in Johnson’s proclamation.” (183)

Amnesty and breaking tradition

Specifically Johnson’s amnesty policies. When compared with the amnesty offered by Lincoln are vastly more expansive, and less punishing to the parties at fault.

To be clear, there is an argument to be made that Lincoln, at least at the very beginning may have been in favor of universal amnesty: “I will here add, that if our success should thus be realized, followed by such desired results, I cannot see, if universal amnesty is granted, how, under the circumstances, I can avoid exacting in return universal suffrage.” Collected Works of Abraham Lincoln. Volume 7. TO JAMES S. WADSWORTH [January, 1864?].  New York Tribune, September 26, 1865; Scribner's Magazine, January, 1893. University of Michigan Library

It is made clear in this quotation that even if Lincoln had supported the type of universal amnesty offered by Johnson, he would have also insisted on universal suffrage. Something that Johnson was starkly opposed to throughout his term.

However, when comparing the actual official actions of both men, we can see that Johnson take little action toward meeting the goals stated by Lincoln, such as the protection of Black people. He also took more liberties as to the extent of his power, and excersized more control over Congress while providing less consequences for rebel actors.

Johnson's amnesty policy was very nearly all encompassing, though there were some excecptions, including government members and people who abandoned their posts or positions to aid in the rebellion. But even those had the opportunity to appeal: “That special application may be made to the President for pardon by any person belonging to the excepted classes, and such clemency will be liberally extended as may be consistent with the facts of the case and the peace and dignity of the United States.”Andrew Johnson, Proclamation 134—Granting Amnesty to Participants in the Rebellion, with Certain Exceptions Online by Gerhard Peters and John T. Woolley, The American Presidency Project. In stark contrast, Lincoln's early amnesty policy was very limited, offering amnesty to only to few rebels, specifically those who had declared their loyalty to the Union while the war was still in progress.

Lincoln left the power to Congress to deide what qualifications would be needed in deciding who could be elected as a delegate, Johnson set the terms himself. And made it so they only needed to take the an oath of loyalty in order to qualify.

Lincoln’s Proclamation had specific aims to protect Black soldiers, Johnson’s was intentionally more vague, providing no specific protections to those who were mistreated due to their race.

Tension with congress

Though Congress did not match the promises made in terms of bettering conditions for Black citizens, they did pass several measures that helped Black people. Including the Civil Rights Act and Reconstruction Amendments. Andrew Johnson on the other hand “did everything in his power to obstruct their implementation.” (334) 

Freedmen’s Bureau

Johnson's actions with the Freedmen's Bureau were especially influencial. In spite of the many vetos he enacted, and their being overwritten by Congress, he was able to cause irreversable harm to the Freedmen's Bureau. 

“As Trescot noted, Johnson’s actions, especially Circular 15, ‘changed radically the character of the Bureau.’ Gone was the idea of settling any considerable portion of the freedmen on land.” (161)
“Johnson had in effect abrogated the Confiscation Act and unilaterally amended the law creating the Bureau.” (161) 
Freedmen's Bureau was able to provide some aid, but: “By the summer, Andrew Johnson – a political and social conservative, opposed to any federal assistance to freed people – had ordered that all confiscated land be returned to Confederates who swore loyalty to the United States and promised to respect the Constitution.” (12)
Johnson’s veto of the bill to extend the life of Bureau essentially killed the idea of allowing freedmen to own the Sherman land in South Carolina. “Blocked efforts to validate the Sherman land titles.” (163)

Civil Rights Act

Though the Civil Rights act did end up passing through Congress, Johnson's veto of it shows not only his prejudices, but his tense relationship with Congress.

“Like his rejection of the Freedmen’s Bureau Bill, Johnson’s veto message repudiated not merely the specific terms of the Civil Rights Bill, but the entire principle behind it.” (250)
“Progressives in Congress – commonly called the “Radical Republicans” for their audacious devotion to expanding democracy and ensuring equality – responded with the Civil Rights bill of 1866, granting citizenship to anyone born in the United States, including African Americans… But President Johnson vetoed the civil rights bill, basing his objections on the misleading claim that the legislation gave Black people “safeguards” whites did not enjoy.” (14)
“It is manifestly and avowedly the object of these laws to confer upon Negroes the privilege of voting and to disfranchise such a number of white citizens as will give the former a clear majority at all elections in the Southern States.”Andrew Johnson, Third Annual Message Online by Gerhard Peters and John T. Woolley, The American Presidency Project https://www.presidency.ucsb.edu/node/201999

In many ways, that veto signified the unwillingness Johnson had to work with and compromise with Congress.
“For Republican moderates, the Civil Rights veto ended all hope of cooperation with the President.” (250)

Once the veto was received, the response was very strong. Though a few in Congress agreed with the President's assessment, many more were infuriated. For several representatives, especially those 'Radical Republicans' this was the end of civil collaboration.

“The President, then, I think, is right.” (Mr. Johnson) (1777)
“Despotism would soon be our fate, preceded by anarchy…” (Mr. Johnson) (1778)

“If the President vetoes the Civil Rights bill, I believe we shall be obliged to draw our swords for a fight and throw away the scabbards.” (250) 
“I regret it on the President’s account, because it is calculated to alienate him from those who elevated him to power, and would gladly have rallied around his Administration to sustain him in the principles upon which he was elected. But above all, sir, I regret it for liberty’s sake, to secure which to ourselves and our posterity this Government was founded.” (Mr. Trumbull) (1755)
“but it is the right that the American people should know that the controversy which exists between him [Andrew Johnson] and Congress in reference to this measure is of his own seeking.” (Mr. Trumbull) (1760)
“If the President of the United States can interpose his authority upon a question of this character, and can compel Congress to succumb to his dictation, he is an emperor, a despot, and not a President of the United States.” (1786) (Mr. Wade)
“Sir, whose fault is it that eleven States are not represented? By what fault of theirs is it that twenty-five loyal States which have stood by this Union and by the Constitution are to be deprived of their right to legislate?” (Mr. Trumbull) (1756)

It is important to consider this relationship as we investigate the impacts of Lincoln's death. Though many of his ideas surrounding how to handle Reconstruction remain a mystery, his fairly positive relationship with Congress, and admirable political ability are not. Even apart from the evidence that he would have handled reconstruction differently, he would have been able to have a more positive relationship with Congress, and effect more changes because of their mutual support.

Though, it could also be seen that Johnson's conservative views pushed Congress further left to offset the balance. It is impossible to know if they would have enacted the same legislature under the more amiable Lincoln administration, or if they would have had less motivation without Johnson as a common enemy.

“But the aims of the Fourteenth Amendment can only be understood within the political and ideological context of 1866: the break with the President, the need to find a measure upon which all Republicans could unite, and the growing consensus within the party around the need for strong federal action to protect the freedmen’s rights, short of the suffrage.” (257)
“Despite all its limitations, Congressional Reconstruction was indeed a radical departure, a stunning and unprecedented experiment in interracial democracy.” (278)

It is also important to consider that if a more radical Reconstruction had been implemented, it may have caused even more pushback from the South. Leading to even more brutality and oppression.

“Had they done so, blacks and the nation might have been spared the long nightmare of disenfranchisement, segregation, and racial violence that followed the end of Reconstruction. Or perhaps, even this Reconstruction plan would have aroused violent opposition in the South.” (335)