Data Processing and Business Insight

Project Requirement (Step-by-Step Implementation)

DATASET - https://business.yelp.com/data/resources/open-dataset/

Requirements - (Fully clear implementation with explanation and workings)


Step 1 — Business Question & Topic Framing

The notebook should clearly define the business problem:

Question:
“In a given city, which business categories look most promising to start a new business, and what factors correlate with high customer ratings?”

Expected outputs:

Top 10 “high-opportunity” business categories in the selected city

A short checklist of key success drivers derived from data

A simple predictive model to classify businesses as High Rated vs Not

Step 2 — Input: Import Data from Web Dataset Files

Use the public Yelp Open Dataset

Programmatically read:

business.json

optionally a sampled review.json

Parse JSON line-by-line (streaming approach) to handle large files efficiently

Computing concepts demonstrated:

File I/O

JSON parsing

Functions and modular code

Step 3 — Processing (Part A): Database Storage

Automatically create an SQLite database

Define database schema for business (and review if used)

Create indexes for efficient querying

Insert parsed data into tables programmatically

Concepts demonstrated:

Database creation

Schema design

Indexing

Step 4 — Processing (Part B): Data Warehouse SQL Queries

Run analytical SQL queries such as:

Average star rating by category within a city

Business count per category (competition level)

Categories with high ratings but low competition

Top benchmark businesses within promising categories

Concepts demonstrated:

GROUP BY

HAVING

ORDER BY

Aggregations

Analytics-style SQL queries

Step 5 — Data Structures & Algorithms (Core Logic)

Implement a category opportunity scoring algorithm:

Use dictionaries / Counters to aggregate category metrics

Use heapq (priority queue) to efficiently extract top-K categories

Example logic:

Demand proxy: average stars + average review count

Competition proxy: number of businesses

Combine these into a custom “Opportunity Score”

Concepts demonstrated:

dict, set, list

heap / priority queue

sorting

custom algorithm design

Step 6 — Output: Analytics Model & Visualization

Implement a simple analytics model (low complexity):

Logistic Regression to predict whether stars >= 4

Use features such as:

review_count

number of categories

is_open status

Outputs to display:

Model accuracy and confusion matrix

Feature importance (model coefficients)

2–3 clear plots showing opportunity ranking and insights

Concepts demonstrated:

Analytics model application

Data visualization

Step 7 — Final Deliverables for Class

Jupyter Notebook that runs end-to-end using Run All

No manual inputs, no API keys, no web scraping

A short slide deck (5–7 slides) summarizing:

Project topic and architecture

OOP class design

Database schema and SQL queries

Data structures and algorithms used

Final results and insights

Technical Constraints

Keep implementation simple and robust

Execution time under 1 minute

Focus on clarity and explainability over complexity

Deliverables

One well-commented Jupyter Notebook (.ipynb)

SQLite database created programmatically

Clean visual outputs and final summary section