From Skeptic to Strategist

A Pragmatic Approach to AI in Development

By Shawn Weekly | 20+ Years Software Architecture Experience

The Skeptic: Bias from 20+ Years of Experience

I've been a software architect for over two decades, and I was deeply skeptical of generative AI. I've seen countless "magic bullet" technologies fail to deliver on their promises:

  • The "NoCode" platforms that still required code
  • The "enterprise configurable" systems that were supposed to limit the need for custom programming
  • The "blockchain" solutions looking for problems to solve

My initial view of AI-generated code was equally cynical: it was an unreliable, untraceable black box and a threat to code quality. How could you trust code you didn't write? How could you maintain it? How could you debug it when (not if) it failed?

"I'm not anti-AI. I'm anti-hype. And there is a LOT of hype around AI."

The Turning Point: The Cimbology Research

My research on Cimbology forced me to test these assumptions rigorously. I discovered that AI, used naively, is indeed dangerous:

The "Baseline LLM" Test

When I tested a baseline LLM (GPT-5-Mini) against a Professional Engineering exam with zero context, it achieved 77% accuracy. Impressive, but when I looked deeper, I discovered it was confidently wrong on critical safety questions. It behaved like a "dumb-but-fast" intern who had read every book but understood nothing.

The danger: 77% feels trustworthy. It's passing most tests. But the 23% it gets wrong are the questions that lead to equipment failure, safety violations, and lawsuits.

The "Naive RAG" Disaster

I then implemented "simple RAG" (Retrieval-Augmented Generation), expecting it to improve accuracy. Instead, accuracy dropped to 53%—a 24% decline. This was my "aha moment": adding AI features without understanding how they work makes things worse, not better.

The lesson: AI is not a magic wand. It's a tool that requires expert guidance and quality data that is properly structured and verified.

The Insight: SME-Guided AI

The solution is not to replace the experts, but to multiply them. The true value of AI is unlocked with an "Expert-in-the-Loop" workflow.

AI is the Assistant, not the Architect

AI is a tool for generating boilerplate, refactoring, and exploring options—all under expert supervision. I use GitHub Copilot daily, but I review every line it generates. I use it to accelerate, not to abdicate my responsibility as an architect.

Example: When building Cimbology's API, Copilot generated 80% of the CRUD boilerplate, but I wrote the GraphRAG orchestration logic by hand. I also authored many specifications that drove Copilot. That's the core process; I provide the expertise and expert guidance, but I DIDN'T write boilerplate.

Context is King

The Cimbology project proved that an AI's value is directly proportional to the quality of the context you provide. This is why my architecture focuses on GraphRAG that provides verifiable, structured knowledge from a Knowledge Graph, not just a blind vector search.

The proof: Simple RAG (bad context) = 53% accuracy. Advanced RAG (good context) = 83% accuracy. KG-Enhanced RAG (great context) = 85% accuracy. Same LLM. Different context.

Use AI for Acceleration, not Abdication

I use AI daily to save dozens of hours. The key is knowing what to ask, how to ask it, and how to verify the output. You don't "talk" to AI; you direct it.

My workflow:
1. Define the requirement precisely (write a spec, user story, or punchlist)
2. Use AI to generate the first draft
3. Review, refine, and test rigorously
4. Own the final result. AI can't be responsible for the code, it's MY code.

What I Bring to Your Team

I don't just use AI; I understand how to deploy it responsibly within an enterprise. I can teach your development teams how to adopt a mature, efficient process that uses AI to accelerate timelines without sacrificing a single bit of engineering quality or code control.

Proven Methodology

My Cimbology research isn't theoretical, it's a working system with measurable results. I've proven that GraphRAG can achieve high accuracy with full traceability. I can bring this same rigor to your knowledge management challenges.

Hands-On Leadership

I don't just design architectures on whiteboards. I write code, configure infrastructure, and ship working systems. I've built and deployed full-stack solutions end-to-end, from database schema to frontend UI.

Training & Enablement

I can teach your team the "smart intern" and "expert review" process that makes AI safe to use in production. This includes:

  • How to write effective prompts (precision, not chat)
  • How to structure code reviews for AI-generated code
  • How to measure and validate AI accuracy (test sets, not hunches)
  • How to avoid the "naive RAG" trap I fell into

Domain Expertise

I've spent years deep in the electrical utility CIM standard. I built dotTC57, an open-source .NET library for IEC 61970/61968. I understand your domain, your data models, and your challenges. I'm not a generic "AI consultant", I'm a utility industry technologist who happens to know a good bit about AI, because I've been fighting with it (and winning) for two+ years.

Tools I Use Daily

My philosophy isn't abstract, here are the specific AI tools I use every day to turbo-charge my productivity:

GitHub Copilot

For code generation, boilerplate, and exploring API patterns. I use it for ~80% of CRUD logic, ~40% of business logic, and ~10% of architectural code.

Key lesson: It's best at "known patterns" (CRUD, REST APIs). Terrible at novel architectures (like GraphRAG orchestration).

Semantic Kernel

Microsoft's AI orchestration framework. I use it for prompt engineering, LLM provider abstraction, and plugin-based extensibility in Cimbology.

Why I chose it: No vendor lock-in. Works with Azure OpenAI, Google AI, local models, etc.

Claude / ChatGPT / Gemini

For architecture exploration, documentation writing, and complex problem-solving. I use long-form prompts (like punchlists and architecture specifications) to get precise results.

Key lesson: The more context you provide, the better the output. "Chatting" with AI is inefficient and dangerous, but "directing" it is powerful.

VS Code + Copilot

My daily IDE setup. VS Code with Github Copilot is excellent for "AI pair programming" where you want the AI to see your entire codebase context.

Productivity gain: ~3-4 hours saved daily on boilerplate, refactoring, and test writing.

Let's Talk About Responsible AI in Your Organization

I'm seeking opportunities to apply this expertise in a technical leadership role. If you're looking for someone who can implement AI systems with rigor, transparency, and measurable results, let's connect.