Flexible json converter #20
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CSV portion finished, json portion not yet started
Unfinished, untested
json converter actually unfinished
Remove json->json conversion pipeline, add json schemas as separate files, extract saving function from CDC data downloader to be available universally
Add option to not re-build metadata, move schemas into their own folder
`save_data()` previously would overwrite files in a directory if it received a new file with the same name. This addition blocks it from doing so and alerts users without failing the script.
Extra logic for `json_converter.py`, a logging context manager for `json_converter.py`, small helper function changes, and a `README.md` in the `scripts/` directory for guides on how to use the scripts
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request:
json_converter.py) that converts, validates, and saves.csvfiles to the RespiLens JSON formatmetadata_builder.py) and saving (save_respilens_data.py) functionality fromprocess_cdc_data.pyto be available as scripts in theRespiLens/scripts/directoryscripts/schemas/location-metadata.schema.jsonandscripts/schemas/respilens-data.schema.jsonJSON schema files, to use for validating location metadata and RespiLens dataREADME.mdfile in thescripts/directory, for useful info on how to utilize the scriptsThis would address issues #11 and #12.
@jcblemai I have other notes and questions, but those would be easier to address in a call.