Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse data from NY's historical PDFs #477

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jsvine
Copy link

@jsvine jsvine commented May 17, 2022

Responding to the call-out here: #476

This being my first commit to the project, and not knowing how the maintainers would like to handle the overlap between the data sources, I tried to take the least destructive approach.

And I think I obeyed the Contribution guidelines, but don't hesitate to holler if I haven't.

Also, in order to get the pre-commit hook to pass, I had to upgrade two dependencies in .pre-commit-config.yaml, due to changes in psf/black. (Cf.: psf/black#2966 and adamchainz/blacken-docs#141.) Let me know if there's another way you'd like that handled.

Responding to the call-out here:
biglocalnews#476

This being my first commit to the project, and not knowing how the
maintainers would like to handle the overlap between the data sources, I
tried to take the least destructive approach.
warn/scrapers/ny.py Outdated Show resolved Hide resolved
warn/scrapers/ny.py Outdated Show resolved Hide resolved
@jsvine
Copy link
Author

jsvine commented May 25, 2022

Added a commit addressing your (very reasonable) requests 👍

@palewire
Copy link
Contributor

Great. Thank you. Do we have any overlap and duplication between the PDFs and our other data sources?

@jsvine
Copy link
Author

jsvine commented May 26, 2022

Looks like there's quite a bit of overlap, both in actual data and timeframes. I don't know enough, however, about the history/provenance of the non-PDF data files to suggest which source(s) should take precedence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants