DataLens - Intelligent Data Analysis Tool

A powerful, modern web application for analyzing CSV files and SQLite databases entirely in your browser. DataLens provides comprehensive data analysis, visualization, and transformation capabilities with a beautiful, premium user interface. All processing happens client-side - your data never leaves your browser.

🚀 Built entirely with AI in just 30 minutes - Demonstrating the power of modern AI-assisted development!

✨ Features

📁 File Management

Multi-file Support: Open and manage multiple CSV or database files simultaneously
File Tabs: Easy switching between open files with visual indicators
Database Table Selection: When opening SQLite databases with multiple tables, choose which table to analyze
Persistent Storage: Files and analysis state are saved in browser storage (IndexedDB + localStorage)
File Types Supported:
- CSV files (.csv)
- SQLite databases (.db, .sqlite, .sqlite3)

📊 Overview Dashboard

Smart Insights: Automatic data quality analysis with closable notifications
- Data completeness percentage
- Duplicate row detection
- Column type recommendations
- Large dataset optimization alerts
Statistics Cards: Key metrics at a glance
- Total rows and columns
- Numeric vs text column counts
- Data quality score
Data Quality Metrics:
- Empty cells count and percentage
- Duplicate rows detection
- Data completeness score
- Column type distribution
Column Summary Grid: Quick overview of all columns with type indicators

📈 Numeric Analysis

Column Selection: Choose any numeric column for detailed analysis
Comprehensive Statistics:
- Minimum, Maximum, Average, Median
- Standard Deviation
- Sum of all values
- Valid vs Null value counts
- Outlier detection (IQR method)
Interactive Visualizations:
- Trend line chart showing values over rows
- Bar chart for first 20 values
- Distribution histogram with adjustable bins (10-100)
Performance Optimized: Automatic sampling for large datasets (>5000 rows)

📝 Text Analysis

Column Selection: Analyze any text column in detail
Key Statistics:
- Unique values count
- Null values count
- Average text length
- Total word count
Pattern Detection: Automatically identifies common patterns in text data
Top Values: Most frequent values with occurrence counts
Detailed Analysis (Collapsible Sections):
- Length analysis (min, max, average)
- Character statistics (letters, digits, special characters)
- Word analysis (unique words, average words per value)
- Value distribution (single vs multiple occurrences)

📋 All Columns View

Complete Column Overview: Grid view of all columns
Per-Column Information:
- Column name and type (numeric/text)
- Unique value count
- Null value count
- For numeric columns: min, max, average

🔗 Correlations Analysis

Column Pair Selection: Choose two numeric columns to analyze
Correlation Metrics:
- Pearson correlation coefficient
- Regression line equation
Interactive Scatter Plot:
- Visual representation of relationship
- Regression line overlay
- Optimized rendering for large datasets
Color-Coded Correlation Strength:
- Strong (>0.7): Red indicator
- Moderate (0.4-0.7): Yellow indicator
- Weak (<0.4): Gray indicator

📊 Advanced Data Table

Global Search: Search across all columns simultaneously
Column-Specific Filters:
- Numeric filters: Supports >, <, >=, <=, ranges (e.g., 10-20), and exact values
- Text filters: Substring search and comma-separated OR logic
- Smart column detection for filter types
- Column selector for datasets with >6 columns
Sorting: Click column headers to sort (ascending/descending)
Pagination:
- Customizable page size (10, 20, 50, 100 rows)
- Navigation controls (first, previous, next, last)
- Page counter display
Column Actions:
- Copy all column values to clipboard
- Export filtered data to CSV
Responsive Design: Horizontal scrolling for wide tables
Sticky Headers: Column headers remain visible while scrolling

🔧 Data Transformations

Rename Columns: Change column names while preserving data
Type Conversion: Convert columns between:
- Auto-detect
- Number
- Text
- Date
- Boolean
Delete Columns: Remove unwanted columns from the dataset
In-Memory Only: Transformations are applied immediately but don't modify original files

🎨 User Interface

Premium Design: Compact, professional layout inspired by enterprise applications
Dark Mode: Full dark theme support with system preference detection
Sticky Navigation:
- Header stays fixed at top
- Page tabs stick below header when scrolling
- File tabs scroll away naturally
- Glassmorphism effect with backdrop blur
Responsive Layout: Works on desktop, tablet, and mobile devices
Smooth Animations: Smooth transitions and hover effects

⚡ Performance

Optimized for Large Datasets:
- Automatic sampling for charts (>5000 rows)
- Efficient single-pass statistics calculation
- Lazy loading of heavy computations
Client-Side Processing: All analysis happens in the browser
IndexedDB Storage: Large datasets stored efficiently in browser IndexedDB
Memory Efficient: Smart data structures and optimized algorithms

🚀 Getting Started

Prerequisites

Node.js 16+ and npm/yarn/pnpm
Modern web browser with WebAssembly support

Installation

Clone or download the repository
Install dependencies:

npm install
# or
yarn install
# or
pnpm install

Build the application:

npm run build

Open the application:
- Recommended: Open dist/index-standalone.html directly in your browser (works offline, no server needed)
- Alternative: Run npm run serve and open the provided URL

Development Mode

Run the development server with hot reload:

npm run dev

Then open your browser to http://localhost:5173 (or the port shown in terminal)

📖 Usage Guide

Uploading Files

Click the upload area or drag and drop a CSV or database file
For SQLite databases with multiple tables, a modal will appear to select which table to open
Files are automatically saved to browser storage and persist across sessions

Navigating the Interface

File Tabs (top bar): Switch between open files, close files, or add new files
Page Tabs (below file tabs): Switch between different analysis views
- Overview: General statistics and insights
- Numeric Analysis: Detailed numeric column analysis
- Text Analysis: Text column pattern detection
- All Columns: Complete column overview
- Correlations: Relationship analysis between columns
- Data Table: Interactive data table with filters

Using the Data Table

Search: Type in the global search box to filter rows across all columns
Filter Columns: Use column-specific filter boxes
- For numeric columns: Use operators like >100, <50, 10-20, or exact values
- For text columns: Type substring or comma-separated terms (OR logic)
Sort: Click column headers to sort ascending/descending
Copy Column: Hover over a column header and click the copy icon
Export: Click "Export" to download filtered data as CSV
Pagination: Use controls at bottom to navigate pages

Transforming Data

Click the "Transform" button above the content area
Rename Column:
- Select a column
- Enter new name
- Click "Rename"
Convert Type:
- Select a column
- Choose target type
- Click "Convert"
Delete Column:
- Find the column in the delete section
- Click the X button
- Confirm deletion

Analyzing Numeric Columns

Go to "Numeric Analysis" tab
Select a column from the dropdown
View comprehensive statistics and charts
Adjust distribution bins for histogram (10-100 bins)

Analyzing Text Columns

Go to "Text Analysis" tab
Select a text column from the dropdown
View patterns, top values, and detailed statistics
Expand collapsible sections for more details

Correlation Analysis

Go to "Correlations" tab
Select two numeric columns
View correlation coefficient and scatter plot
See regression line equation

🏗️ Architecture

Technology Stack

React 18: Modern UI framework with hooks
Vite: Fast build tool and dev server
Tailwind CSS: Utility-first CSS framework
PapaParse: Robust CSV parsing library
sql.js: SQLite compiled to WebAssembly for browser database reading
TanStack Table: Powerful table component with sorting, filtering, pagination
Recharts: Beautiful, responsive chart library
Lucide React: Modern icon library

Project Structure

data_viewer/
├── src/
│   ├── components/
│   │   ├── dashboard/          # Dashboard sub-components
│   │   │   ├── CollapsibleSection.jsx
│   │   │   ├── StatsCards.jsx
│   │   │   └── TextAnalysis.jsx
│   │   ├── Dashboard.jsx       # Main dashboard component
│   │   ├── DataTable.jsx       # Advanced data table
│   │   ├── DataTransform.jsx   # Data transformation tools
│   │   ├── FileTabs.jsx        # File management tabs
│   │   ├── Tabs.jsx            # Page navigation tabs
│   │   ├── SmartInsights.jsx   # Data quality insights
│   │   └── ...
│   ├── contexts/
│   │   └── ThemeContext.jsx   # Dark mode context
│   ├── utils/
│   │   ├── fileParser.js       # CSV/DB parsing
│   │   ├── statsOptimized.js   # Statistical calculations
│   │   ├── storage.js          # Browser storage management
│   │   └── textAnalysis.js     # Text pattern detection
│   ├── App.jsx                 # Main application component
│   └── main.jsx                # Application entry point
├── package.json
├── vite.config.js
└── tailwind.config.js

Data Flow

File Upload → fileParser.js parses CSV/DB
Data Storage → Saved to IndexedDB for large files, localStorage for metadata
Analysis → statsOptimized.js calculates statistics
Display → Components render data and visualizations
Transformations → Applied in-memory, can be saved back to storage

Storage Strategy

Small Files (<5MB): Stored in IndexedDB
Large Files: Data stored in IndexedDB, metadata in localStorage
State Persistence: Active file, active tab, and file list saved automatically
Privacy: All data stays in browser, never sent to servers

🎯 Key Features Explained

Smart Insights

Automatically analyzes your data and provides actionable insights:

Data Quality: Warns if completeness <80%, celebrates if >95%
Duplicates: Alerts if duplicate rate >10%
Column Types: Suggests using Numeric/Text tabs for detailed analysis
Performance: Notifies about optimizations for large datasets
Dismissible: Click X to close insights (persists across sessions)

Advanced Filtering

The data table supports sophisticated filtering:

Numeric Filters:

Comparison: >100, <50, >=10, <=200
Ranges: 10-20, 100..200, 5:15
Exact: 42

Text Filters:

Substring: john finds "john", "johnny", "johnson"
Multiple terms: apple,orange,banana (OR logic)

Performance Optimizations

Sampling: Charts use sampling for datasets >5000 rows
Lazy Analysis: Heavy text analysis only runs when needed
Efficient Algorithms: Single-pass statistics calculation
Debounced Search: Global search debounced to 300ms

🌐 Browser Compatibility

✅ Chrome/Edge (latest)
✅ Firefox (latest)
✅ Safari (latest)
✅ Any modern browser with:
- WebAssembly support (for SQLite)
- ES6+ JavaScript support
- IndexedDB support

📦 Deployment

Static Hosting (Recommended)

This app is a static site and can be deployed anywhere:

Vercel

npm i -g vercel
vercel

Netlify

npm i -g netlify-cli
npm run build
netlify deploy --prod --dir=dist

GitHub Pages

Build: npm run build
Push dist contents to gh-pages branch
Enable GitHub Pages in repo settings

Other Options

Cloudflare Pages
AWS S3 + CloudFront
Firebase Hosting
Surge.sh
Any static file host

Single-File Build

The build process creates dist/index-standalone.html which includes all assets inlined. This single file:

Works with file:// protocol
No server needed
Can be shared as a single HTML file
Perfect for offline use

🔒 Privacy & Security

100% Client-Side: All processing happens in your browser
No Network Requests: Data never sent to any server
Local Storage Only: Files stored in your browser's IndexedDB
No Tracking: Zero analytics or tracking code
Open Source: Full source code available for inspection

🛠️ Development

Available Scripts

npm run dev - Start development server
npm run build - Build for production
npm run build:server - Build without single-file bundle
npm run preview - Preview production build
npm run serve - Serve built files locally

Building

The build process:

Bundles all JavaScript and CSS
Optimizes assets
Creates single-file standalone version
Outputs to dist/ folder

📝 License

MIT License - feel free to use, modify, and distribute

🙏 Acknowledgments

Built with:

DataLens - Intelligent data analysis, entirely in your browser. 🔍✨

Name		Name	Last commit message	Last commit date
src		src
.gitignore		.gitignore
README.md		README.md
build-single-file.js		build-single-file.js
index.html		index.html
package.json		package.json
postcss.config.js		postcss.config.js
serve.js		serve.js
tailwind.config.js		tailwind.config.js
vite.config.js		vite.config.js
yarn.lock		yarn.lock

Latest commit

History

Folders and files

Repository files navigation