I get asked this a lot. Most often its folks with two kids who think they’re overwhelmed. Sometimes its folks with three.; rarely they have one.
My response depends on the questioner’s situation. I actually found 4 a bit easier than 3. And, we have one older than the twins so I never landed at 2. There seems to be a huge psychological jump to go from two to three kids, but we never faced that issue.
The world is definitely set up for families of 4. From a “world” perspective the main difference for us was that we needed a minivan with 4 as we’d gotten away with 3 across the back seat previously. Most other things don’t care if you have 5 or 6. Typical hotel rooms can be an issue since they won’t give you 2 trundle beds, but we strongly prefer Embassy Suites so we don’t have to go to bed when the kids do and their standard room sleeps 6. We have 7 total and that just opens an entirely different can of worms as no place is set up for that.
The logistics of 4 do get to be a lot (see post from Sept 10, 2014). It’s similar to having 3 singletons since the twins do many things together. You have to be very organized about your time and do much more planning. And, I have had to lean on other parents more often when they have to be in different places simultaneously. I also sometimes wonder about the age gap (8y top to bottom for us) for family activities. But, when I had 3 preschoolers I found it nice to have another over for a playdate because they would pair off, regardless of who they actually came to play with. We had real 3rd wheel syndrome otherwise. As our 4th got older this became the standard dynamic and it gave me much more freedom to do my thing vs being on entertainment duty.
I do get weird looks when I have them all out myself. Nobody expects that many kids to go shopping anywhere, for example. And, I really got noticed when I took the older 4 to the carnival. But, I got lots of looks with the twins so I’m just used to it by now.