Lisbon over Lagos?

Going to South of Portugal next November with our baby.

We are landing in Lisbon and staying there for 3 fulls days (5 total but 1st and last aren't full). Then, renting a car for Lagos for 3 fulls days (still 5 days but 1st and last day aren't full)

We are wondering if we add 1 one day for Lisbon and cut 1 day in Lagos. What do you think? Our pace might be slower with the baby so wondering if staying another full day in Lisbon is beneficial.

After Lagos, we are driving to Seville than coming to Faro than Lisbon.