The word quietly suggests that the person you're talking to misunderstands reality, and that it's up to you - the arbiter of truth - to correct that error.
Carolyn Kopprasch, chief happiness officer of the social media service Buffer, explained her argument against "actually" in a blog post, which we first spotted on Lifehacker:
It almost doesn't matter how good the news is; if it comes after "actually," I feel like I was somehow wrong about something.
Consider these two sentences:
Actually, you can do this under "Settings."
Sure thing, you can do this under "Settings!" :)
While it's a subtle change, taking out "actually" changes the way status is established in a conversation.
When you employ "actually," you're asserting that you're the knowledgeable, dominant person who ought to be deferred to - adding the slightest sense of competition to the exchange. Whether customer, friend, or colleague, the person you're talking to might feel belittled.
But if you sidestep "actually," as Kopprasch's "sure thing" does, you're suggesting that both people are working together to tackle a problem - adding a sense of cooperation to the exchange and avoiding belittlement.
This is key.
To paraphrase Maya Angelou, people will forget what you say, but they'll remember how you made them feel.
There are tons of incentives for making people feel good. From what social science tells us, it's one of the best investments we can make.