unrecognized token

This error occurs when SQLite encounters a character or sequence it doesn't understand. Often caused by special characters, wrong quotes, or encoding issues.

The unrecognized token error means SQLite found a character sequence it can't parse.

Understanding the Error

Error: unrecognized token: "@"
Error: unrecognized token: "$"

The character or symbol isn't valid SQL syntax.

Common Causes

1. Special Characters

SQL

-- Wrong: @ is not valid in this context
SELECT * FROM users WHERE email = user@example.com;

-- Correct: use quotes
SELECT * FROM users WHERE email = 'user@example.com';

2. Wrong Quote Types

SQL

-- Wrong: curly/smart quotes (from word processors)
SELECT * FROM users WHERE name = 'Alice';

-- Correct: straight quotes
SELECT * FROM users WHERE name = 'Alice';

3. Variable Syntax From Other Languages

SQL

-- Wrong: PHP-style variables
SELECT * FROM users WHERE id = $id;

-- Wrong: Ruby-style interpolation
SELECT * FROM users WHERE id = #{id};

-- Correct: use parameters
SELECT * FROM users WHERE id = ?;
SELECT * FROM users WHERE id = :id;

4. Encoding Issues

Non-ASCII characters causing problems:

JAVASCRIPT

// Hidden characters in copy-pasted SQL
const sql = 'SELECT * FROM users';  // Might contain invisible chars

5. Line Ending Issues

Windows vs Unix line endings in queries.

How to Fix It

Solution 1: Use Proper Quotes

SQL

-- String values: single quotes
SELECT * FROM users WHERE name = 'Alice';

-- Identifiers (if needed): double quotes or brackets
SELECT * FROM "user table" WHERE name = 'Alice';
SELECT * FROM [user table] WHERE name = 'Alice';

Solution 2: Use Parameters

JAVASCRIPT

// Always use parameters for values
db.get('SELECT * FROM users WHERE email = ?', ['user@example.com']);

// Named parameters
db.get('SELECT * FROM users WHERE email = :email', { email: 'user@example.com' });

Solution 3: Clean the SQL String

JAVASCRIPT

function cleanSQL(sql) {
  return sql
    // Replace smart quotes with straight quotes
    .replace(/['']/g, "'")
    .replace(/[""]/g, '"')
    // Remove invisible characters
    .replace(/[\u200B-\u200D\uFEFF]/g, '')
    // Normalize line endings
    .replace(/\r\n/g, '\n')
    .replace(/\r/g, '\n');
}

Solution 4: Check Character Encoding

JAVASCRIPT

// Log actual bytes to find hidden chars
console.log('SQL:', sql);
console.log('Bytes:', Buffer.from(sql));
console.log('Chars:', [...sql].map(c => c.charCodeAt(0)));

Solution 5: Type SQL Instead of Copy-Paste

Copy-pasting from documents, websites, or PDFs often introduces invisible or wrong characters.

Common Token Issues

| Token | Issue | Solution | |-------|-------|----------| | @ | Email addresses | Use quotes: 'email@test.com' | | $ | Variable syntax | Use ? or :name parameters | | # | Comments (some DBs) | Use -- or /* */ | | \ | Escape char | Double it: \\ | | ; | Statement end | Use only at end | | ' ' | Smart quotes | Use ' straight quote |

Best Practices

Use parameterized queries for all values
Type SQL manually instead of copy-paste
Use a SQL editor that shows invisible chars
Check encoding if SQL comes from files
Validate user input before building queries