Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added scripts for csv to json conversion #105

Merged
merged 1 commit into from
Oct 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
# misc
.DS_Store
*.pem
/data/scripts/*.csv
/data/scripts/*.json

# debug
npm-debug.log*
Expand Down
11 changes: 11 additions & 0 deletions data/scripts/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
How to use:

1. Create a csv file for the desired conversion. See the sample for row format. For example, if converting rentcaps from csv to json, create a file that follows the format in "rentcap_sample", name it "rentcap.csv", and place it in this folder.

2. Run the desired conversion script, for example:

> python csv_to_json_rentcap.py

3. This will result in a json file. (In this example, "rentcap.json".) Replace the appropriate file in the /data/ directory.

4. Merge changes and redeploy the site.
7 changes: 7 additions & 0 deletions data/scripts/csv_to_json_elig.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import pandas as pd
df = pd.read_csv('eligibility.csv', dtype={'min_units': 'Int32', 'landlord_occupancy_exemption_units': 'Int32'}).set_index('geography')
json = df.to_json(orient='index', indent=2).replace(':', ': ')

f = open('eligibility.json', 'w')
f.write(json)
f.close()
7 changes: 7 additions & 0 deletions data/scripts/csv_to_json_moratoria.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import pandas as pd
df = pd.read_csv('moratoria.csv').set_index('geography')
json = df.to_json(orient='index', indent=2).replace(':', ': ')

f = open('moratoria.json', 'w')
f.write(json)
f.close()
7 changes: 7 additions & 0 deletions data/scripts/csv_to_json_rent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import pandas as pd
df = pd.read_csv('rentcap.csv').set_index('id')
json = df.to_json(orient='index', indent=2).replace(':', ': ')

f = open('rentcap.json', 'w')
f.write(json)
f.close()
29 changes: 29 additions & 0 deletions data/scripts/eligibility_sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
geography,reit_corp_llc,construction_date,min_units,sec8,lihtc,duplex,dorms,senior_care,hotels,hotels_q2,landlord_shared_exemption,relative_occupancy_q1,relative_occupancy_q2,relative_occupancy_wh,landlord_occupancy_tenancy,landlord_occupancy,landlord_occupancy_1year,berkeley_duplex,duplex_attached,duplex_building
Alameda,,1995-02-02,2,exempt,eligible,eligible,exempt,exempt,31,,exempt,,,,,,,,,
Antioch,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,31,,exempt,,exempt,,2,,,,,
Baldwin Park,,1995-01-01,2,eligible,eligible,eligible,exempt,exempt,31,,,,,,,2,,,,
Bell Gardens,,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,31,,,,,,3,,,,,
Berkeley,,1980-07-01,2,eligible,eligible,eligible,exempt,exempt,14,,exempt,,,,,2,,exempt,,
Beverly Hills,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,31,,,,,,,,,,,
Commerce,,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,2,,,,,
Culver City,,1995-02-02,2,exempt,exempt,eligible,exempt,exempt,31,,,,,,,,,,,
East Palo Alto,,1988-01-02,2,exempt,exempt,eligible,exempt,exempt,31,,exempt,,,,,,3,,,
Fairfax,eligible,1995-02-02,2,eligible,exempt,eligibile,exempt,exempt,30,,,,,,,,,,,
Hayward,,1979-07-02,2,exempt,exempt,eligible,exempt,exempt,30,,,,,,,2,,,,
Inglewood,,,2,exempt,exempt,eligible,exempt,exempt,31,,exempt,,,,,2,,,,exempt
Los Angeles,,1978-10-02,2,eligible,exempt,eligible,exempt,exempt,31,,,,,,,,,,,
Unincorporated LA County,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,31,,,,,,,,,,,
Los Gatos,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,,30,,,,,,,,,,
Mountain View,,1995-02-02,3,eligible,eligible,exempt,exempt,exempt,,30,,,,,,,,,,
Oakland,,1983-01-01,2,exempt,exempt,eligible,exempt,exempt,31,,exempt,,,,,,,,,
Oxnard,,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,,29,,,,,2,,,,exempt,
Pasadena,eligible,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,,,,,,
Pomona,eligible,1995-02-02,2,exempt,exempt,eligible,eligible,eligible,,,exempt,,,,,,,,,
Richmond,,1995-02-02,2,exempt,exempt,eligible,exempt,exempt,,14,exempt,,,,,,,,,
Sacramento,,1995-02-02,2,exempt,exempt,eligible,exempt,exempt,,30,exempt,exempt,,,,,,,,
San Francisco,,1979-06-14,2,exempt,exempt,eligible,exempt,exempt,32,,,,,,,,,,,
San Jose,,1979-09-07,3,exempt,exempt,exempt,exempt,exempt,31,,,,,,,,,,,exempt
Santa Ana,,,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,,,,,exempt,
Santa Monica,,1979-04-11,2,eligible,eligible,eligible,exempt,exempt,,14,,,,,,3,,,,
West Hollywood,,1979-07-02,2,eligible,exempt,eligible,exempt,exempt,,30,exempt,,,exempt,,,,,,
Statewide,eligible,,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,2,,,,,
9 changes: 9 additions & 0 deletions data/scripts/moratoria_sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
geography,start,end,max_increase
Alameda,2020-04-30,2022-06-30,0.0000
Bell Gardens,2022-04-25,2022-10-12,0.0000
Beverly Hills,2020-03-15,2022-06-30,0.0000
Los Angeles,2020-03-20,2024-01-31,0.0000
Richmond,2020-04-21,2021-09-30,0.0000
San Jose,2020-04-28,2020-12-31,0.0000
Unincorporated LA County,2020-03-04,2022-12-31,0.0000
West Hollywood,2020-04-06,2022-03-01,0.0000
165 changes: 165 additions & 0 deletions data/scripts/rentcap_sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
id,geography,area,start_year,start,end,max_increase
ala2024,Alameda,,2024,2024-09-01,2025-08-31,0.0270
ala2023,Alameda,,2023,2023-09-01,2024-08-31,0.0294
ala2022,Alameda,,2022,2022-09-01,2023-08-31,0.0350
ala2021,Alameda,,2021,2021-09-01,2022-08-31,0.0270
ala2020,Alameda,,2020,2020-09-01,2021-08-31,0.0100
ala2019,Alameda,,2019,2019-09-01,2020-08-31,0.0280
ant2024_4,Antioch,,2024,2024-07-12,2024-09-10,0.0192
ant2024_3,Antioch,,2024,2024-05-13,2024-07-11,0.0228
ant2024_2,Antioch,,2024,2024-03-12,2024-05-12,0.0144
ant2024_1,Antioch,,2024,2024-01-11,2024-03-11,0.0156
ant2023_6,Antioch,,2023,2023-11-14,2024-01-10,0.0168
ant2023_5,Antioch,,2023,2023-09-14,2023-11-13,0.0204
ant2023_4,Antioch,,2023,2023-07-13,2023-09-13,0.0174
ant2023_3,Antioch,,2023,2023-05-11,2023-07-12,0.0252
ant2023_2,Antioch,,2023,2023-03-15,2023-05-10,0.0300
ant2023_1,Antioch,,2023,2023-01-13,2023-03-14,0.0294
ant2022,Antioch,,2022,2022-08-23,2023-01-12,0.0300
bel2023,Bell Gardens,,2023,2023-11-01,2024-10-31,0.0190
bel2022,Bell Gardens,,2022,2022-10-12,2023-10-31,0.0395
ber2024,Berkeley,,2024,2024-01-01,2024-12-31,0.0190
ber2023,Berkeley,,2023,2023-01-01,2023-12-31,0.0440
ber2022,Berkeley,,2022,2022-01-01,2022-12-31,0.0210
ber2021,Berkeley,,2021,2021-01-01,2021-12-31,0.0100
ber2020,Berkeley,,2020,2020-01-01,2020-12-31,0.0210
ber2019,Berkeley,,2019,2019-01-01,2019-12-31,0.0250
bev2023,Beverly Hills,,2023,2023-07-01,2024-06-30,0.0320
bev2022,Beverly Hills,,2022,2022-07-01,2023-06-30,0.0310
bev2021,Beverly Hills,,2021,2021-07-01,2022-06-30,0.0390
bev2020,Beverly Hills,,2020,2020-07-01,2021-06-30,0.0300
bev2019,Beverly Hills,,2019,2019-07-01,2020-06-30,0.0310
cul2024_5,Culver City,,2024,2024-08-01,2024-09-30,0.0325
cul2024_4,Culver City,,2024,2024-05-01,2024-07-31,0.0300
cul2024_3,Culver City,,2024,2024-04-01,2024-04-30,0.0325
cul2024_2,Culver City,,2024,2024-02-01,2024-03-31,0.0350
cul2024_1,Culver City,,2024,2024-01-01,2024-01-31,0.0400
cul2023_3,Culver City,,2023,2023-12-01,2023-12-31,0.0425
cul2023_2,Culver City,,2023,2023-11-01,2023-11-30,0.0475
cul2023_1,Culver City,,2023,2023-01-01,2023-10-31,0.0500
cul2022_5,Culver City,,2022,2022-05-01,2022-12-31,0.0500
cul2022_4,Culver City,,2022,2022-04-01,2022-04-30,0.0450
cul2022_3,Culver City,,2022,2022-03-01,2022-03-31,0.0400
cul2022_2,Culver City,,2022,2022-02-01,2022-02-28,0.0350
cul2022_1,Culver City,,2022,2022-01-01,2022-01-31,0.0300
cul2021_3,Culver City,,2021,2021-12-01,2021-12-31,0.0250
cul2021_2,Culver City,,2021,2021-10-01,2021-11-30,0.0225
cul2021_1,Culver City,,2021,2021-01-01,2021-09-30,0.0200
cul2020,Culver City,,2020,2020-10-30,2020-12-31,0.0225
cul2019,Culver City,,2019,2019-08-12,2020-10-29,0.0300
epa2024,East Palo Alto,,2024,2024-07-01,2025-06-30,0.0190
epa2023,East Palo Alto,,2023,2023-07-01,2024-06-30,0.0420
epa2022,East Palo Alto,,2022,2022-07-01,2023-06-30,0.0420
epa2021,East Palo Alto,,2021,2021-07-01,2022-06-30,0.0130
epa2020,East Palo Alto,,2020,2020-07-01,2021-06-30,0.0230
epa2019,East Palo Alto,,2019,2019-07-01,2020-06-30,0.0280
ff2024,Fairfax,,2024,2024-09-01,2025-08-31,0.0285
ff2023,Fairfax,,2023,2023-09-01,2024-08-31,0.0252
ff2022,Fairfax,,2022,2022-12-02,2023-08-31,0.0300
hay2024,Hayward,,2024,2024-01-01,2024-12-31,0.0500
hay2023,Hayward,,2023,2023-01-01,2023-12-31,0.0500
hay2022,Hayward,,2022,2022-01-01,2022-12-31,0.0500
hay2021,Hayward,,2021,2021-01-01,2021-12-31,0.0500
hay2020,Hayward,,2020,2020-01-01,2020-12-31,0.0500
hay2019,Hayward,,2019,2019-01-01,2019-12-31,0.0500
ing2024,Inglewood,,2024,2024-05-01,2025-04-30,0.0390
ing2023,Inglewood,,2023,2023-05-01,2024-04-30,0.0790
ing2022,Inglewood,,2022,2022-05-01,2023-04-30,0.0518
ing2021,Inglewood,,2021,2021-06-09,2022-04-30,0.0300
ing2020,Inglewood,,2020,2020-10-01,2021-06-08,0.0300
ing2019,Inglewood,,2019,2019-12-05,2020-09-30,0.0319
la2024_2,Los Angeles,,2024,2024-07-01,2025-06-30,0.0400
la2024_1,Los Angeles,,2024,2024-02-01,2024-06-30,0.0400
la2023,Los Angeles,,2023,2023-07-01,2024-01-31,0.0000
la2022,Los Angeles,,2022,2022-07-01,2023-06-30,0.0500
la2021,Los Angeles,,2021,2021-07-01,2022-06-30,0.0300
la2020,Los Angeles,,2020,2020-07-01,2021-06-30,0.0300
la2019,Los Angeles,,2019,2019-07-01,2020-06-30,0.0400
lac2020,Unincorporated LA County,,2020,2020-04-01,2022-12-31,0.0000
mtv2024,Mountain View,,2024,2024-09-01,2025-08-31,0.0240
mtv2023,Mountain View,,2023,2023-09-01,2024-08-31,0.0500
mtv2022,Mountain View,,2022,2022-09-01,2023-08-31,0.0500
mtv2021,Mountain View,,2021,2021-09-01,2022-08-31,0.0200
mtv2020,Mountain View,,2020,2020-09-01,2021-08-31,0.0290
mtv2019,Mountain View,,2019,2019-09-01,2020-08-31,0.0350
oak2024,Oakland,,2024,2024-08-01,2025-07-31,0.0230
oak2023,Oakland,,2023,2023-08-01,2024-07-31,0.0250
oak2022,Oakland,,2022,2022-08-01,2023-07-31,0.0300
oak2021,Oakland,,2021,2021-07-01,2022-07-31,0.0190
oak2020,Oakland,,2020,2020-07-01,2021-06-30,0.0270
oak2019,Oakland,,2019,2019-07-01,2020-06-30,0.0350
oxn2022,Oxnard,,2022,2022-06-04,2030-12-31,0.0400
pas2023,Pasadena,,2023,2023-10-01,2024-09-30,0.0275
pom2023,Pomona,,2023,2023-08-01,2024-07-31,0.0400
pom2022,Pomona,,2022,2022-08-01,2023-07-31,0.0400
ric2024,Richmond,,2024,2024-09-01,2025-08-31,0.0140
ric2023,Richmond,,2023,2023-09-01,2024-08-31,0.0300
ric2022,Richmond,,2022,2022-09-01,2023-08-31,0.0520
ric2021,Richmond,,2021,2021-09-01,2022-08-31,0.0160
ric2020,Richmond,,2020,2020-09-01,2021-08-31,0.0290
ric2019,Richmond,,2019,2019-09-01,2020-08-31,0.0350
sac2024,Sacramento,,2024,2024-07-01,2025-06-30,0.0880
sac2023,Sacramento,,2023,2023-07-01,2024-06-30,0.0920
sac2022,Sacramento,,2022,2022-07-01,2023-06-30,0.1000
sac2021,Sacramento,,2021,2021-07-01,2022-06-30,0.0900
sac2020,Sacramento,,2020,2020-07-01,2021-06-30,0.0600
sac2019,Sacramento,,2019,2019-09-12,2020-06-30,0.0830
sf2024,San Francisco,,2024,2024-03-01,2025-02-28,0.0170
sf2023,San Francisco,,2023,2023-03-01,2024-02-29,0.0360
sf2022,San Francisco,,2022,2022-03-01,2023-02-28,0.0230
sf2021,San Francisco,,2021,2021-03-01,2022-02-28,0.0070
sf2020,San Francisco,,2020,2020-03-01,2021-02-28,0.0180
sf2019,San Francisco,,2019,2019-03-01,2020-02-29,0.0260
sj2024,San Jose,,2024,2024-01-01,2024-12-31,0.0500
sj2023,San Jose,,2023,2023-01-01,2023-12-31,0.0500
sj2022,San Jose,,2022,2022-01-01,2022-12-31,0.0500
sj2021,San Jose,,2021,2021-01-01,2021-12-31,0.0500
sj2020,San Jose,,2020,2020-01-01,2020-12-31,0.0500
sj2019,San Jose,,2019,2019-01-01,2019-12-31,0.0500
sa2024,Santa Ana,,2024,2024-09-01,2025-08-31,0.0300
sa2023,Santa Ana,,2023,2023-09-01,2024-08-31,0.0254
sa2022,Santa Ana,,2022,2022-09-01,2023-08-31,0.0300
sm2024,Santa Monica,,2024,2024-09-01,2025-08-31,0.0300
sm2023,Santa Monica,,2023,2023-09-01,2024-08-31,0.0280
sm2022_2,Santa Monica,,2022,2023-02-01,2023-08-31,0.0080
sm2022_1,Santa Monica,,2022,2022-09-01,2023-01-31,0.0600
sm2021,Santa Monica,,2021,2021-09-01,2022-08-31,0.0165
sm2020,Santa Monica,,2020,2020-09-01,2021-08-31,0.0143
sm2019,Santa Monica,,2019,2019-09-01,2020-08-31,0.0203
wh2024,West Hollywood,,2024,2024-09-01,2025-08-31,0.0300
wh2023_2,West Hollywood,,2023,2023-09-01,2024-08-31,0.0250
wh2023_1,West Hollywood,,2023,2023-03-01,2023-08-31,0.0300
wh2022,West Hollywood,,2022,2022-09-01,2023-02-28,0.0600
wh2021,West Hollywood,,2021,2021-09-01,2022-08-31,0.0300
wh2020,West Hollywood,,2020,2020-09-01,2021-08-31,0.0075
wh2019,West Hollywood,,2019,2019-09-01,2020-08-31,0.0250
casf2024,Statewide,Oakland-Hayward-San_Francisco,2024,2024-08-01,2025-07-31,0.0880
casf2023,Statewide,Oakland-Hayward-San_Francisco,2023,2023-08-01,2024-07-31,0.0920
casf2022,Statewide,Oakland-Hayward-San_Francisco,2022,2022-08-01,2023-07-31,0.1000
casf2021,Statewide,Oakland-Hayward-San_Francisco,2021,2021-08-01,2022-07-31,0.0880
casf2020,Statewide,Oakland-Hayward-San_Francisco,2020,2020-08-01,2021-07-31,0.0610
casf2019,Statewide,Oakland-Hayward-San_Francisco,2019,2019-08-01,2020-07-31,0.0900
casd2024,Statewide,San_Diego-Carlsbad,2024,2024-08-01,2025-07-31,0.0880
casd2023,Statewide,San_Diego-Carlsbad,2023,2023-08-01,2024-07-31,0.0920
casd2022,Statewide,San_Diego-Carlsbad,2022,2022-08-01,2023-07-31,0.1000
casd2021,Statewide,San_Diego-Carlsbad,2021,2021-08-01,2022-07-31,0.0900
casd2020,Statewide,San_Diego-Carlsbad,2020,2020-08-01,2021-07-31,0.0600
casd2019,Statewide,San_Diego-Carlsbad,2019,2019-08-01,2020-07-31,0.0830
cariv2024,Statewide,Riverside-San_Bernardino-Ontario,2024,2024-08-01,2025-07-31,0.0880
cariv2023,Statewide,Riverside-San_Bernardino-Ontario,2023,2023-08-01,2024-07-31,0.0920
cariv2022,Statewide,Riverside-San_Bernardino-Ontario,2022,2022-08-01,2023-07-31,0.1000
cariv2021,Statewide,Riverside-San_Bernardino-Ontario,2021,2021-08-01,2022-07-31,0.0900
cariv2020,Statewide,Riverside-San_Bernardino-Ontario,2020,2020-08-01,2021-07-31,0.0600
cariv2019,Statewide,Riverside-San_Bernardino-Ontario,2019,2019-08-01,2020-07-31,0.0830
cala2024,Statewide,Los_Angeles-Long_Beach-Anaheim,2024,2024-08-01,2025-07-31,0.0890
cala2023,Statewide,Los_Angeles-Long_Beach-Anaheim,2023,2023-08-01,2024-07-31,0.0880
cala2022,Statewide,Los_Angeles-Long_Beach-Anaheim,2022,2022-08-01,2023-07-31,0.1000
cala2021,Statewide,Los_Angeles-Long_Beach-Anaheim,2021,2021-08-01,2022-07-31,0.0860
cala2020,Statewide,Los_Angeles-Long_Beach-Anaheim,2020,2020-08-01,2021-07-31,0.0570
cala2019,Statewide,Los_Angeles-Long_Beach-Anaheim,2019,2019-08-01,2020-07-31,0.0830
cagen2024,Statewide,Rest_Of_California,2024,2024-08-01,2025-07-31,0.0880
cagen2023,Statewide,Rest_Of_California,2023,2023-08-01,2024-07-31,0.0920
cagen2022,Statewide,Rest_Of_California,2022,2022-08-01,2023-07-31,0.1000
cagen2021,Statewide,Rest_Of_California,2021,2021-08-01,2022-07-31,0.0900
cagen2020,Statewide,Rest_Of_California,2020,2020-08-01,2021-07-31,0.0600
cagen2019,Statewide,Rest_Of_California,2019,2019-08-01,2020-07-31,0.0830
Loading