Working With AI Output

AI can write code quickly, but speed without quality is dangerous. Learning to effectively review and iterate on AI output is one of the most important skills in AI development.

The Review Process

Never blindly copy-paste AI-generated code. Follow this review workflow:

1. Read Before Running: Understand what the code does
2. Check the Logic: Can you explain it to someone else?
3. Verify APIs Exist: AI sometimes hallucinates methods
4. Look for Issues: Check against common AI mistakes
5. Test Thoroughly: Don't assume it works
6. Iterate: Refine through conversation

The Data: AI Code Has More Issues

Research from CodeRabbit analyzing 470 pull requests found:

This doesn't mean AI code is bad—it means it requires careful review.

Common AI Mistakes

Hallucinated APIs

AI sometimes suggests methods or packages that don't exist.

// AI might suggest this
import { fetchUser } from 'next-auth/helpers'  // This doesn't exist!

// Always verify by checking documentation
import { getSession } from 'next-auth/react'   // This is real

19.7% of npm packages suggested by AI don't exist. Always verify.

Missing Edge Cases

AI often handles the happy path but misses edge cases:

// AI might write
function getFirstItem(array) {
  return array[0]
}

// But what about empty arrays?
function getFirstItem(array) {
  if (!array || array.length === 0) {
    return undefined
  }
  return array[0]
}

Outdated Syntax

AI training data can be outdated:

// AI might suggest class components
class UserProfile extends React.Component {
  render() { /* ... */ }
}

// Modern approach uses function components
function UserProfile() {
  return /* ... */
}

Security Vulnerabilities

AI code is more likely to contain security issues:

// ❌ Vulnerable to SQL injection
const query = `SELECT * FROM users WHERE id = ${userId}`

// ✅ Use parameterized queries
const query = 'SELECT * FROM users WHERE id = $1'
await db.query(query, [userId])

Performance Anti-Patterns

Watch for unnecessary operations:

// ❌ AI might nest loops inefficiently
users.forEach(user => {
  const role = roles.find(r => r.userId === user.id)  // O(n²)
})

// ✅ Use a map for O(n) lookup
const roleMap = new Map(roles.map(r => [r.userId, r]))
users.forEach(user => {
  const role = roleMap.get(user.id)  // O(1) lookup
})

Code Review Checklist

Use this checklist for AI-generated code:

Correctness

• Does the code solve the actual problem?
• Are all edge cases handled?
• Are error states managed properly?
• Does it match the expected behavior?

Security

• No hardcoded secrets or API keys?
• Input validation present?
• No SQL/XSS injection vulnerabilities?
• Proper authentication/authorization?

Quality

• Do all referenced APIs/libraries exist?
• Does naming follow project conventions?
• Is the code maintainable?
• No unnecessary complexity?

Performance

• No excessive I/O operations?
• Appropriate data structures used?
• No obvious performance anti-patterns?

Iterative Refinement

AI rarely produces perfect code on the first try. Plan to iterate:

You: Create a user authentication function

AI: [provides code]

You: Good start, but add rate limiting for failed attempts

AI: [updated code]

You: Now add logging for security auditing

AI: [improved code]

You: The rate limiter should reset after successful login

AI: [final code]

When to Accept

Accept AI code when:

• You understand what it does
• It follows your project patterns
• It handles edge cases
• It passes your tests
• It's secure and performant

When to Reject or Refine

Reject or ask for changes when:

• You don't fully understand it
• It uses unfamiliar libraries
• It's overly complex
• It ignores your constraints
• It introduces security risks

Debugging AI Code

When AI code doesn't work:

1. Isolate the Problem

Find the specific failing part:

// Add logging to understand execution flow
console.log('Step 1: Input received', input)
const processed = processData(input)
console.log('Step 2: After processing', processed)

2. Ask AI to Explain

Explain what this code does line by line:
[paste the problematic code]

I expected [behavior] but got [actual result]

3. Verify External Dependencies

Check that all APIs, packages, and methods actually exist by consulting documentation.

4. Create Minimal Reproduction

Simplify to the smallest case that fails:

// Instead of debugging the entire feature
// Isolate just the failing part
const test = myFunction(simpleInput)
console.log(test) // What do we actually get?

5. Feed Back Results

Tell the AI what happened:

I tried your solution, but I'm getting this error:
[new error message]

The code now looks like:
[updated code]

What should I try next?

The 70% Problem

AI can often get you 70% of the way very quickly—but the last 30% can take longer than expected.

Plan for:

• Edge case handling
• Error states
• Performance optimization
• Security hardening
• Integration with existing code

Don't be surprised when the "quick" AI solution needs significant refinement.

Building Verification Habits

Write Tests First

Consider writing tests before asking AI for implementation:

describe('calculateDiscount', () => {
  it('applies 10% for orders over $100', () => {
    expect(calculateDiscount(150)).toBe(15)
  })

  it('returns 0 for orders under $100', () => {
    expect(calculateDiscount(50)).toBe(0)
  })

  it('handles edge case of exactly $100', () => {
    expect(calculateDiscount(100)).toBe(0)
  })
})

Then ask AI: "Implement a function that passes these tests."

Use TypeScript

TypeScript catches many AI mistakes at compile time:

// TypeScript will catch this
function processUser(user: User): string {
  return user.naem  // Error: Property 'naem' does not exist
}

Run Linters and Formatters

Automated tools catch issues AI might introduce:

# ESLint catches code quality issues
npm run lint

# TypeScript catches type errors
npm run type-check

Summary

• Never trust blindly: AI code has 1.7x more issues than human code
• Check for common mistakes: Hallucinated APIs, missing edge cases, outdated syntax
• Use the checklist: Correctness, security, quality, performance
• Iterate: AI output improves through conversation
• Plan for the 70%: The last 30% often takes longer than expected

Next Steps

Now that you can effectively review AI output, let's set up version control to safely track your changes and collaborate with AI tools.