Skip to content

Endpoints for Building Permits Data Processes - #68

Open
LilLizDog wants to merge 24 commits into
mainfrom
feature/endpoints_for_housing_data
Open

Endpoints for Building Permits Data Processes#68
LilLizDog wants to merge 24 commits into
mainfrom
feature/endpoints_for_housing_data

Conversation

@LilLizDog

@LilLizDog LilLizDog commented Dec 1, 2025

Copy link
Copy Markdown
Collaborator

Brief description of the feature implemented:
In this pull request, I created query endpoints for Building Permit data from stlouis_building_permit table (columns: id, created_on, data_posted_on, is_active, and necessary columns) that filters to active records

The acceptance criteria:

  1. Endpoints: GET /api/building.
  2. Inline docstrings, handle large responses (pagination).
  3. Unit/integration tests with sample building permit records (verify is_active logic).
  4. Secure endpoints with rate limiting stub.
  5. Document the building permit site in PR (e.g., "Building Permit API - Site: https://www.stlouis-mo.gov/data/dashboards/building-permits/neighborhoods.cfm") and add the URL here in the issue update.
  6. Map data_posted_on to site publish dates, deactivate old entries (is_active=0) when new housing stats are scraped.

Context for design and implementation decisions:
I decided to use direct SQL queries instead of ORM models for simplicity. I also decided to have the tests run with the method below since I couldn't get a generic code that would work for everyone without going the method below. The tests world run properly with my personal postgres password hardcoded into the code, but when I tried to use a tagging method for the password, it wouldn't work. So I settled for the below method.

Code quality self-assessment including testing approaches:
The code is very easy to read, has in-depth comments and documentation, and fills the criteria in a simple and straightforward manner. I probably could have figured out a bettwer method to run the tests than with the method below, but I didn't feel like wasting any more hours on that one little problem when my tech lead could probably solve it in 5 minutes. There are 3 tests that test_building_permits_endpoints.py tests. These test:
That GET /api/building should return only rows where is_active = TRUE.
That pagination params page + page_size must control result size.
That rate limiting stub should not block requests (no 429).

To run tests, type:
If using Git Bash or WSL:
export PG_PASSWORD="yourRealPasswordHere"
export PG_HOST="localhost"
export PG_USER="postgres"
export PG_DB="stl_data"
export PG_PORT="5432"
pytest tests/read_service/test_building_permits_endpoint.py -q

If using Windows CMD:
set PG_PASSWORD=yourRealPasswordHere
set PG_HOST=localhost
set PG_USER=postgres
set PG_DB=stl_data
set PG_PORT=5432
pytest tests/read_service/test_building_permits_endpoint.py -q

If using PowerShell:
$env:PG_PASSWORD="yourRealPasswordHere"
$env:PG_HOST="localhost"
$env:PG_USER="postgres"
$env:PG_DB="stl_data"
$env:PG_PORT="5432"
pytest tests/read_service/test_building_permits_endpoint.py -q

All tests pass.

Documentation of any refactoring or technical decisions made:
I was having the same issue with this sprint as I had with the last sprint: I couldn't get the code to work without hardcoding my personal postgres password into it. However, I figured out that if you run the tests with the method above, you don't need to have your password hardcoded into the code.
Also note that data_posted_on corresponds to the publish/update date on that dashboard. When new housing stats are scraped for a neighborhood, write-service will mark previous rows is_active = 0 and insert the new row with is_active = 1. GET /api/building only returns is_active = 1 rows and supports pagination via page and page_size.

Active discussion with code reviewers and testers:
I haven't had any discussion with code reviewers and testers yet since I am still waiting on my tech lead to review my code.

Evidence of incorporating feedback from code review:
I haven't been able to incorporate any feedback from code reviews into my code since I am still waiting on my tech lead to review my code.

Building Permit API Site:
https://www.stlouis-mo.gov/data/dashboards/building-permits/neighborhoods.cfm](https://www.stlouis-mo.gov/data/dashboards/building-permits/neighborhoods.cfm?utm_source=chatgpt.com)

@premkiran2

premkiran2 commented Dec 4, 2025

Copy link
Copy Markdown
Collaborator

@LilLizDog Table is getting created successfully and sample data is getting added
Extend the feature to work on real data from official website https://www.stlouis-mo.gov/
And try to pass all the test cases

@premkiran2

premkiran2 commented Dec 7, 2025

Copy link
Copy Markdown
Collaborator

#68 (comment) Refer this comment.
Check the pipeline CI / Tests (pytest) - Python 3.11 (pull_request), You can find errors,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants