Help me budget 4 days between L.A., San Francisco and a train ride between them

I'm a lifelong East Coast resident who has never been west of the rockies and I am wrapping up a conference in L.A. This was a grueling experience and I haven't had a single second to myself. I have until Sunday to get back home and wanted to also see San Francisco. I'm staying in Downtown Los Angeles for now and I do not have a car. Originally I was intending on heading out to San Francisco the very minute the conference is over, but now I feel like it is wrong to leave without taking a peak at America's 2nd largest city (I virtually haven't). That being said, I have heard it's almost impossible to do that without a car. FWIW I am the kind of traveller that doesnt really do sights. Just the REALLY big ones and otherwise goes for the nice walkable neighborhoods that look beautiful. Not big on nature either.

I am also told it's a crime not to take the Amtrak between the two cities because it's the most beautiful train ride in the country.

If I spend a day in L.A. to myself and take the train, I'd have only 2 days in San Francisco.

So it boils down to three questions:

1) Is L.A. (and surrounding areas like Beverly Hills, Venice, Santa Monica) worth tackling in one day?

2) Is the train between L.A. and SF an unmissable experience worth losing a day to? (Otherwise I'd fly)

3) is downtown SF compact enough to do in 2 days or so?