An open-button sweater can look almost like a jacket, provided it's the appropriate type. We recommend a fine knit with a shawl collar, which can most closely approximate the look of a traditional suit jacket.
If you're feeling underdressed, add a tie that can hide underneath the front buttons of your sweater, just like it would on a jacket. Some manufacturers even sell "sweater blazers," which are exactly what they sound like: sweaters that are fashioned to look like blazers.