Lexical Analyser

A lexical analyser (tokenizer) built from scratch in C — the first phase of a compiler. Reads a C source file character by character and classifies every token into one of 7 types with color-coded terminal output.

Token Types

Type	Color	Examples
Keyword	Blue	int, if, while, return
Identifier	Cyan	main, lineNumber, i
Constant	Green	42, 0xFF, 0b1010, 3.14
String Literal	White	"hello world"
Operator	Yellow	+, ==, &&, ++, ->
Symbol	Default	; , { } ( )
Unknown/Error	Red	123G, 0b112, 089

Features

• Reads C source file character by character using fgetc()
• Skips preprocessor directives (#include, #define)
• Handles single-line (//) and multi-line (/* */) comments
• Detects hexadecimal (0x), binary (0b), octal (leading 0), decimal numbers
• Flags invalid literals with line number (e.g. 0b1012, 089, 123G)
• Detects 2-character operators: ==, !=, <=, >=, &&, ||, ++, --, ->
• Tracks brace balance for { } ( ) [ ] — reports mismatches
• Color-coded output using ANSI escape codes

Technologies Used

• Language: C
• Concepts: File handling, fgetc, ungetc, enums, structs, string processing

How to Compile

make

How to Run

./lexer input.c

Sample Output

Tokens found:
----------------
Keyword      : int
Identifier   : main
Symbol       : (
Keyword      : int
Identifier   : argc
Symbol       : ,
...
Error at line 5: Invalid number literal '0b1012'
----------------

Project Structure

File	Purpose
main.c	Entry point, token loop, color printing
lexer.c	Core engine — getNextToken()
lexer.h	Token struct, TokenType enum, declarations
keyword.c	32 C keywords table and isKeyword()
keyword.h	isKeyword() declaration
error.c	Brace balance tracking and mismatch detection
error.h	Error function declarations