6 Questions to Ask Before You Buy Any AI App (Because Vibe Coding Will Bite You in the Ass)

An AI-built app looks great in the office and falls apart in the field. Six questions every window and door company should ask before buying "AI-powered" software.

#ArtificialIntelligence#SoftwareEvaluation#Fenestration

Here's the honest truth, especially for those of us running big quotes, custom orders and real customers.

First, what "vibe coding" means. You tell an AI what you want in plain English, it writes the code behind the scenes, slaps a nice-looking screen on top, and you hope it holds together.

I'm not against it. I use it every day. But vibe coding with no developer behind it is like anchoring an opening with drywall screws and hoping the inspector doesn't look.

80% in Ten Minutes. The Last 20% Is Where It Dies.

In ten minutes you can tell an AI, "Build me a quoting tool that calculates square footage, adds my markup and spits out an invoice." It hands you a clean, shiny screen. You plug in a few test sizes, the math works, and you feel like you just outsmarted every software developer on Earth.

That gets you about 80% of the way. It looks great on a laptop in an air-conditioned office.

Then you hand it to a sales rep in the field.

  • The cell signal dies. Your rep walks into a concrete building or a gated community with no bars. A lot of these apps just spin, choke and lose the whole takeoff. Real field software saves on the device and syncs later, so your rep doesn't look like an amateur in somebody's living room.
  • The plumbing is ugly. AI doesn't care about clean, long-term engineering. It cares about making you happy right now. Ask it to move a button or change a font and it might quietly rewire your math, and now your tax or lineal-foot calculation is wrong on page four.
  • Nobody knows how to fix it. If you don't understand how the thing was built and the AI gets stuck on an error, you're dead in the water. You can't tell a homeowner or a GC waiting on a big custom package that your prompt is having a nervous breakdown.

The BS Detector: 6 Questions for Any Vendor Selling "AI-Powered" Software

Every vendor pitching contractors this year has slapped "AI" all over the deck. A lot of the time, they took an old feature, bolted a basic chatbot onto it, raised the monthly price and called it revolutionary.

Before you sign an annual contract or hand an untested app to your reps, sit the vendor down and ask these. If they start stammering, waving their hands or burying you in jargon, show them the door.

1. "What AI is actually doing the work?"

  • Good answer: They name the models they use (Claude, GPT, Gemini, whatever), explain how they connect to them, and tell you which parts are AI and which parts are plain code.
  • BS answer: "Our proprietary neural network operates natively across our cloud ecosystem." Translation: they don't want you to know.

2. "Does the AI touch my pricing, or does real code do the math?"

  • Dealbreaker: Never let AI do raw pricing math. These models predict the next likely word or number. They're not calculators. If AI is figuring your lineal footage, glass weight or design pressures on the fly, sooner or later it'll make up a discount or underprice a big job.
  • What you want to hear: "AI reads the notes, pulls the text and organizes the takeoff. Fixed formulas handle every penny of pricing and tax."

3. "What happens when my guy is in a concrete condo with zero bars?"

  • Reality check: A lot of these apps need to talk to the vendor's server every time you tap a button.
  • The test: Your rep is measuring openings on the 14th floor of a beach tower with no signal. Does the app save to the device and sync later, or does it throw an error and wipe out an hour of work? No real offline mode means it's not built for the field.

4. "Is my customer and pricing data training somebody's public AI?"

  • The risk: If your client lists, discount tiers and takeoffs run through consumer-grade AI accounts, you don't really know where your margins end up.
  • What you want in writing: Business-grade AI agreements with zero data retention, meaning the AI provider doesn't keep or train on your data. Get it on paper, not on a sales call.

5. "If the AI gets a product code or spec wrong, who pays for the reorder?"

  • The accountability test: Watch how fast the rep gets nervous. If the software fills out factory order sheets, ask what stops a mistake from going through.
  • What you need: A human review step. Good AI gets you the info fast. It never sends a custom order to a manufacturer without a real estimator signing off.

6. "Show me the export button. Can I get my data out tomorrow?"

  • The trap: Shady vendors make it easy to put your data in and hard to get it out. Cancel six months later and you get unreadable junk or a "migration fee."
  • The requirement: One-click export to clean spreadsheet files (CSV or Excel) for every estimate, contact and line item. Make them show you, not tell you.

The Bottom Line

There's always another shiny toy next month. Some kid online will claim he built a whole business system on his lunch break, the forums will catch fire, and people will panic.

Don't buy the circus. Tools like Claude and Grok Bot are serious equipment. Learn to use them and you'll have a real edge over the guys still running on legal pads and dry-erase boards.

Use AI to cut the grunt work and move faster. Just treat it like a power tool, not a magic wand. If the foundation under your business isn't solid, all the good vibes in the world won't keep the roof up.

Thoughts?

ABOUT THE AUTHOR

John Stephens

Founder, Everything Windows and Doors

The founder started Everything Windows and Doors after years in the South Florida impact window and door trade. He writes about the journey here in the EWD Journal.

Get started

Ready to Stop Collecting Software & Start Building Real Workflows?

Direct operational discussions for dealership principals looking to eliminate margin bleed and automate field-to-desk relays.