Skip to content

Commit

Permalink
added scripts for csv to json conversion
Browse files Browse the repository at this point in the history
  • Loading branch information
matt committed Sep 17, 2024
1 parent f9b7334 commit 9ca45e8
Show file tree
Hide file tree
Showing 8 changed files with 237 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
# misc
.DS_Store
*.pem
/data/scripts/*.csv
/data/scripts/*.json

# debug
npm-debug.log*
Expand Down
11 changes: 11 additions & 0 deletions data/scripts/README.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
How to use:

1. Create a csv file for the desired conversion. See the sample for row format. For example, if converting rentcaps from csv to json, create a file that follows the format in "rentcap_sample", name it "rentcap.csv", and place it in this folder.

2. Run the desired conversion script, for example:

> python csv_to_json_rentcap.py

3. This will result in a json file. (In this example, "rentcap.json".) Replace the appropriate file in the /data/ directory.

4. Merge changes and redeploy the site.
7 changes: 7 additions & 0 deletions data/scripts/csv_to_json_elig.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import pandas as pd
df = pd.read_csv('eligibility.csv', dtype={'min_units': 'Int32', 'landlord_occupancy_exemption_units': 'Int32'}).set_index('geography')
json = df.to_json(orient='index', indent=2).replace(':', ': ')

f = open('eligibility.json', 'w')
f.write(json)
f.close()
7 changes: 7 additions & 0 deletions data/scripts/csv_to_json_moratoria.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import pandas as pd
df = pd.read_csv('moratoria.csv').set_index('geography')
json = df.to_json(orient='index', indent=2).replace(':', ': ')

f = open('moratoria.json', 'w')
f.write(json)
f.close()
7 changes: 7 additions & 0 deletions data/scripts/csv_to_json_rent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import pandas as pd
df = pd.read_csv('rentcap.csv').set_index('id')
json = df.to_json(orient='index', indent=2).replace(':', ': ')

f = open('rentcap.json', 'w')
f.write(json)
f.close()
29 changes: 29 additions & 0 deletions data/scripts/eligibility_sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
geography,reit_corp_llc,construction_date,min_units,sec8,lihtc,duplex,dorms,senior_care,hotels,hotels_q2,landlord_shared_exemption,relative_occupancy_q1,relative_occupancy_q2,relative_occupancy_wh,landlord_occupancy_tenancy,landlord_occupancy,landlord_occupancy_1year,berkeley_duplex,duplex_attached,duplex_building
Alameda,,1995-02-02,2,exempt,eligible,eligible,exempt,exempt,31,,exempt,,,,,,,,,
Antioch,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,31,,exempt,,exempt,,2,,,,,
Baldwin Park,,1995-01-01,2,eligible,eligible,eligible,exempt,exempt,31,,,,,,,2,,,,
Bell Gardens,,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,31,,,,,,3,,,,,
Berkeley,,1980-07-01,2,eligible,eligible,eligible,exempt,exempt,14,,exempt,,,,,2,,exempt,,
Beverly Hills,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,31,,,,,,,,,,,
Commerce,,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,2,,,,,
Culver City,,1995-02-02,2,exempt,exempt,eligible,exempt,exempt,31,,,,,,,,,,,
East Palo Alto,,1988-01-02,2,exempt,exempt,eligible,exempt,exempt,31,,exempt,,,,,,3,,,
Fairfax,eligible,1995-02-02,2,eligible,exempt,eligibile,exempt,exempt,30,,,,,,,,,,,
Hayward,,1979-07-02,2,exempt,exempt,eligible,exempt,exempt,30,,,,,,,2,,,,
Inglewood,,,2,exempt,exempt,eligible,exempt,exempt,31,,exempt,,,,,2,,,,exempt
Los Angeles,,1978-10-02,2,eligible,exempt,eligible,exempt,exempt,31,,,,,,,,,,,
Unincorporated LA County,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,31,,,,,,,,,,,
Los Gatos,,1995-02-02,2,eligible,eligible,eligible,exempt,exempt,,30,,,,,,,,,,
Mountain View,,1995-02-02,3,eligible,eligible,exempt,exempt,exempt,,30,,,,,,,,,,
Oakland,,1983-01-01,2,exempt,exempt,eligible,exempt,exempt,31,,exempt,,,,,,,,,
Oxnard,,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,,29,,,,,2,,,,exempt,
Pasadena,eligible,1995-02-02,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,,,,,,
Pomona,eligible,1995-02-02,2,exempt,exempt,eligible,eligible,eligible,,,exempt,,,,,,,,,
Richmond,,1995-02-02,2,exempt,exempt,eligible,exempt,exempt,,14,exempt,,,,,,,,,
Sacramento,,1995-02-02,2,exempt,exempt,eligible,exempt,exempt,,30,exempt,exempt,,,,,,,,
San Francisco,,1979-06-14,2,exempt,exempt,eligible,exempt,exempt,32,,,,,,,,,,,
San Jose,,1979-09-07,3,exempt,exempt,exempt,exempt,exempt,31,,,,,,,,,,,exempt
Santa Ana,,,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,,,,,exempt,
Santa Monica,,1979-04-11,2,eligible,eligible,eligible,exempt,exempt,,14,,,,,,3,,,,
West Hollywood,,1979-07-02,2,eligible,exempt,eligible,exempt,exempt,,30,exempt,,,exempt,,,,,,
Statewide,eligible,,2,eligible,exempt,eligible,exempt,exempt,31,,exempt,,,,2,,,,,
9 changes: 9 additions & 0 deletions data/scripts/moratoria_sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
geography,start,end,max_increase
Alameda,2020-04-30,2022-06-30,0.0000
Bell Gardens,2022-04-25,2022-10-12,0.0000
Beverly Hills,2020-03-15,2022-06-30,0.0000
Los Angeles,2020-03-20,2024-01-31,0.0000
Richmond,2020-04-21,2021-09-30,0.0000
San Jose,2020-04-28,2020-12-31,0.0000
Unincorporated LA County,2020-03-04,2022-12-31,0.0000
West Hollywood,2020-04-06,2022-03-01,0.0000
165 changes: 165 additions & 0 deletions data/scripts/rentcap_sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
id,geography,area,start_year,start,end,max_increase
ala2024,Alameda,,2024,2024-09-01,2025-08-31,0.0270
ala2023,Alameda,,2023,2023-09-01,2024-08-31,0.0294
ala2022,Alameda,,2022,2022-09-01,2023-08-31,0.0350
ala2021,Alameda,,2021,2021-09-01,2022-08-31,0.0270
ala2020,Alameda,,2020,2020-09-01,2021-08-31,0.0100
ala2019,Alameda,,2019,2019-09-01,2020-08-31,0.0280
ant2024_4,Antioch,,2024,2024-07-12,2024-09-10,0.0192
ant2024_3,Antioch,,2024,2024-05-13,2024-07-11,0.0228
ant2024_2,Antioch,,2024,2024-03-12,2024-05-12,0.0144
ant2024_1,Antioch,,2024,2024-01-11,2024-03-11,0.0156
ant2023_6,Antioch,,2023,2023-11-14,2024-01-10,0.0168
ant2023_5,Antioch,,2023,2023-09-14,2023-11-13,0.0204
ant2023_4,Antioch,,2023,2023-07-13,2023-09-13,0.0174
ant2023_3,Antioch,,2023,2023-05-11,2023-07-12,0.0252
ant2023_2,Antioch,,2023,2023-03-15,2023-05-10,0.0300
ant2023_1,Antioch,,2023,2023-01-13,2023-03-14,0.0294
ant2022,Antioch,,2022,2022-08-23,2023-01-12,0.0300
bel2023,Bell Gardens,,2023,2023-11-01,2024-10-31,0.0190
bel2022,Bell Gardens,,2022,2022-10-12,2023-10-31,0.0395
ber2024,Berkeley,,2024,2024-01-01,2024-12-31,0.0190
ber2023,Berkeley,,2023,2023-01-01,2023-12-31,0.0440
ber2022,Berkeley,,2022,2022-01-01,2022-12-31,0.0210
ber2021,Berkeley,,2021,2021-01-01,2021-12-31,0.0100
ber2020,Berkeley,,2020,2020-01-01,2020-12-31,0.0210
ber2019,Berkeley,,2019,2019-01-01,2019-12-31,0.0250
bev2023,Beverly Hills,,2023,2023-07-01,2024-06-30,0.0320
bev2022,Beverly Hills,,2022,2022-07-01,2023-06-30,0.0310
bev2021,Beverly Hills,,2021,2021-07-01,2022-06-30,0.0390
bev2020,Beverly Hills,,2020,2020-07-01,2021-06-30,0.0300
bev2019,Beverly Hills,,2019,2019-07-01,2020-06-30,0.0310
cul2024_5,Culver City,,2024,2024-08-01,2024-09-30,0.0325
cul2024_4,Culver City,,2024,2024-05-01,2024-07-31,0.0300
cul2024_3,Culver City,,2024,2024-04-01,2024-04-30,0.0325
cul2024_2,Culver City,,2024,2024-02-01,2024-03-31,0.0350
cul2024_1,Culver City,,2024,2024-01-01,2024-01-31,0.0400
cul2023_3,Culver City,,2023,2023-12-01,2023-12-31,0.0425
cul2023_2,Culver City,,2023,2023-11-01,2023-11-30,0.0475
cul2023_1,Culver City,,2023,2023-01-01,2023-10-31,0.0500
cul2022_5,Culver City,,2022,2022-05-01,2022-12-31,0.0500
cul2022_4,Culver City,,2022,2022-04-01,2022-04-30,0.0450
cul2022_3,Culver City,,2022,2022-03-01,2022-03-31,0.0400
cul2022_2,Culver City,,2022,2022-02-01,2022-02-28,0.0350
cul2022_1,Culver City,,2022,2022-01-01,2022-01-31,0.0300
cul2021_3,Culver City,,2021,2021-12-01,2021-12-31,0.0250
cul2021_2,Culver City,,2021,2021-10-01,2021-11-30,0.0225
cul2021_1,Culver City,,2021,2021-01-01,2021-09-30,0.0200
cul2020,Culver City,,2020,2020-10-30,2020-12-31,0.0225
cul2019,Culver City,,2019,2019-08-12,2020-10-29,0.0300
epa2024,East Palo Alto,,2024,2024-07-01,2025-06-30,0.0190
epa2023,East Palo Alto,,2023,2023-07-01,2024-06-30,0.0420
epa2022,East Palo Alto,,2022,2022-07-01,2023-06-30,0.0420
epa2021,East Palo Alto,,2021,2021-07-01,2022-06-30,0.0130
epa2020,East Palo Alto,,2020,2020-07-01,2021-06-30,0.0230
epa2019,East Palo Alto,,2019,2019-07-01,2020-06-30,0.0280
ff2024,Fairfax,,2024,2024-09-01,2025-08-31,0.0285
ff2023,Fairfax,,2023,2023-09-01,2024-08-31,0.0252
ff2022,Fairfax,,2022,2022-12-02,2023-08-31,0.0300
hay2024,Hayward,,2024,2024-01-01,2024-12-31,0.0500
hay2023,Hayward,,2023,2023-01-01,2023-12-31,0.0500
hay2022,Hayward,,2022,2022-01-01,2022-12-31,0.0500
hay2021,Hayward,,2021,2021-01-01,2021-12-31,0.0500
hay2020,Hayward,,2020,2020-01-01,2020-12-31,0.0500
hay2019,Hayward,,2019,2019-01-01,2019-12-31,0.0500
ing2024,Inglewood,,2024,2024-05-01,2025-04-30,0.0390
ing2023,Inglewood,,2023,2023-05-01,2024-04-30,0.0790
ing2022,Inglewood,,2022,2022-05-01,2023-04-30,0.0518
ing2021,Inglewood,,2021,2021-06-09,2022-04-30,0.0300
ing2020,Inglewood,,2020,2020-10-01,2021-06-08,0.0300
ing2019,Inglewood,,2019,2019-12-05,2020-09-30,0.0319
la2024_2,Los Angeles,,2024,2024-07-01,2025-06-30,0.0400
la2024_1,Los Angeles,,2024,2024-02-01,2024-06-30,0.0400
la2023,Los Angeles,,2023,2023-07-01,2024-01-31,0.0000
la2022,Los Angeles,,2022,2022-07-01,2023-06-30,0.0500
la2021,Los Angeles,,2021,2021-07-01,2022-06-30,0.0300
la2020,Los Angeles,,2020,2020-07-01,2021-06-30,0.0300
la2019,Los Angeles,,2019,2019-07-01,2020-06-30,0.0400
lac2020,Unincorporated LA County,,2020,2020-04-01,2022-12-31,0.0000
mtv2024,Mountain View,,2024,2024-09-01,2025-08-31,0.0240
mtv2023,Mountain View,,2023,2023-09-01,2024-08-31,0.0500
mtv2022,Mountain View,,2022,2022-09-01,2023-08-31,0.0500
mtv2021,Mountain View,,2021,2021-09-01,2022-08-31,0.0200
mtv2020,Mountain View,,2020,2020-09-01,2021-08-31,0.0290
mtv2019,Mountain View,,2019,2019-09-01,2020-08-31,0.0350
oak2024,Oakland,,2024,2024-08-01,2025-07-31,0.0230
oak2023,Oakland,,2023,2023-08-01,2024-07-31,0.0250
oak2022,Oakland,,2022,2022-08-01,2023-07-31,0.0300
oak2021,Oakland,,2021,2021-07-01,2022-07-31,0.0190
oak2020,Oakland,,2020,2020-07-01,2021-06-30,0.0270
oak2019,Oakland,,2019,2019-07-01,2020-06-30,0.0350
oxn2022,Oxnard,,2022,2022-06-04,2030-12-31,0.0400
pas2023,Pasadena,,2023,2023-10-01,2024-09-30,0.0275
pom2023,Pomona,,2023,2023-08-01,2024-07-31,0.0400
pom2022,Pomona,,2022,2022-08-01,2023-07-31,0.0400
ric2024,Richmond,,2024,2024-09-01,2025-08-31,0.0140
ric2023,Richmond,,2023,2023-09-01,2024-08-31,0.0300
ric2022,Richmond,,2022,2022-09-01,2023-08-31,0.0520
ric2021,Richmond,,2021,2021-09-01,2022-08-31,0.0160
ric2020,Richmond,,2020,2020-09-01,2021-08-31,0.0290
ric2019,Richmond,,2019,2019-09-01,2020-08-31,0.0350
sac2024,Sacramento,,2024,2024-07-01,2025-06-30,0.0880
sac2023,Sacramento,,2023,2023-07-01,2024-06-30,0.0920
sac2022,Sacramento,,2022,2022-07-01,2023-06-30,0.1000
sac2021,Sacramento,,2021,2021-07-01,2022-06-30,0.0900
sac2020,Sacramento,,2020,2020-07-01,2021-06-30,0.0600
sac2019,Sacramento,,2019,2019-09-12,2020-06-30,0.0830
sf2024,San Francisco,,2024,2024-03-01,2025-02-28,0.0170
sf2023,San Francisco,,2023,2023-03-01,2024-02-29,0.0360
sf2022,San Francisco,,2022,2022-03-01,2023-02-28,0.0230
sf2021,San Francisco,,2021,2021-03-01,2022-02-28,0.0070
sf2020,San Francisco,,2020,2020-03-01,2021-02-28,0.0180
sf2019,San Francisco,,2019,2019-03-01,2020-02-29,0.0260
sj2024,San Jose,,2024,2024-01-01,2024-12-31,0.0500
sj2023,San Jose,,2023,2023-01-01,2023-12-31,0.0500
sj2022,San Jose,,2022,2022-01-01,2022-12-31,0.0500
sj2021,San Jose,,2021,2021-01-01,2021-12-31,0.0500
sj2020,San Jose,,2020,2020-01-01,2020-12-31,0.0500
sj2019,San Jose,,2019,2019-01-01,2019-12-31,0.0500
sa2024,Santa Ana,,2024,2024-09-01,2025-08-31,0.0300
sa2023,Santa Ana,,2023,2023-09-01,2024-08-31,0.0254
sa2022,Santa Ana,,2022,2022-09-01,2023-08-31,0.0300
sm2024,Santa Monica,,2024,2024-09-01,2025-08-31,0.0300
sm2023,Santa Monica,,2023,2023-09-01,2024-08-31,0.0280
sm2022_2,Santa Monica,,2022,2023-02-01,2023-08-31,0.0080
sm2022_1,Santa Monica,,2022,2022-09-01,2023-01-31,0.0600
sm2021,Santa Monica,,2021,2021-09-01,2022-08-31,0.0165
sm2020,Santa Monica,,2020,2020-09-01,2021-08-31,0.0143
sm2019,Santa Monica,,2019,2019-09-01,2020-08-31,0.0203
wh2024,West Hollywood,,2024,2024-09-01,2025-08-31,0.0300
wh2023_2,West Hollywood,,2023,2023-09-01,2024-08-31,0.0250
wh2023_1,West Hollywood,,2023,2023-03-01,2023-08-31,0.0300
wh2022,West Hollywood,,2022,2022-09-01,2023-02-28,0.0600
wh2021,West Hollywood,,2021,2021-09-01,2022-08-31,0.0300
wh2020,West Hollywood,,2020,2020-09-01,2021-08-31,0.0075
wh2019,West Hollywood,,2019,2019-09-01,2020-08-31,0.0250
casf2024,Statewide,Oakland-Hayward-San_Francisco,2024,2024-08-01,2025-07-31,0.0880
casf2023,Statewide,Oakland-Hayward-San_Francisco,2023,2023-08-01,2024-07-31,0.0920
casf2022,Statewide,Oakland-Hayward-San_Francisco,2022,2022-08-01,2023-07-31,0.1000
casf2021,Statewide,Oakland-Hayward-San_Francisco,2021,2021-08-01,2022-07-31,0.0880
casf2020,Statewide,Oakland-Hayward-San_Francisco,2020,2020-08-01,2021-07-31,0.0610
casf2019,Statewide,Oakland-Hayward-San_Francisco,2019,2019-08-01,2020-07-31,0.0900
casd2024,Statewide,San_Diego-Carlsbad,2024,2024-08-01,2025-07-31,0.0880
casd2023,Statewide,San_Diego-Carlsbad,2023,2023-08-01,2024-07-31,0.0920
casd2022,Statewide,San_Diego-Carlsbad,2022,2022-08-01,2023-07-31,0.1000
casd2021,Statewide,San_Diego-Carlsbad,2021,2021-08-01,2022-07-31,0.0900
casd2020,Statewide,San_Diego-Carlsbad,2020,2020-08-01,2021-07-31,0.0600
casd2019,Statewide,San_Diego-Carlsbad,2019,2019-08-01,2020-07-31,0.0830
cariv2024,Statewide,Riverside-San_Bernardino-Ontario,2024,2024-08-01,2025-07-31,0.0880
cariv2023,Statewide,Riverside-San_Bernardino-Ontario,2023,2023-08-01,2024-07-31,0.0920
cariv2022,Statewide,Riverside-San_Bernardino-Ontario,2022,2022-08-01,2023-07-31,0.1000
cariv2021,Statewide,Riverside-San_Bernardino-Ontario,2021,2021-08-01,2022-07-31,0.0900
cariv2020,Statewide,Riverside-San_Bernardino-Ontario,2020,2020-08-01,2021-07-31,0.0600
cariv2019,Statewide,Riverside-San_Bernardino-Ontario,2019,2019-08-01,2020-07-31,0.0830
cala2024,Statewide,Los_Angeles-Long_Beach-Anaheim,2024,2024-08-01,2025-07-31,0.0890
cala2023,Statewide,Los_Angeles-Long_Beach-Anaheim,2023,2023-08-01,2024-07-31,0.0880
cala2022,Statewide,Los_Angeles-Long_Beach-Anaheim,2022,2022-08-01,2023-07-31,0.1000
cala2021,Statewide,Los_Angeles-Long_Beach-Anaheim,2021,2021-08-01,2022-07-31,0.0860
cala2020,Statewide,Los_Angeles-Long_Beach-Anaheim,2020,2020-08-01,2021-07-31,0.0570
cala2019,Statewide,Los_Angeles-Long_Beach-Anaheim,2019,2019-08-01,2020-07-31,0.0830
cagen2024,Statewide,Rest_Of_California,2024,2024-08-01,2025-07-31,0.0880
cagen2023,Statewide,Rest_Of_California,2023,2023-08-01,2024-07-31,0.0920
cagen2022,Statewide,Rest_Of_California,2022,2022-08-01,2023-07-31,0.1000
cagen2021,Statewide,Rest_Of_California,2021,2021-08-01,2022-07-31,0.0900
cagen2020,Statewide,Rest_Of_California,2020,2020-08-01,2021-07-31,0.0600
cagen2019,Statewide,Rest_Of_California,2019,2019-08-01,2020-07-31,0.0830

0 comments on commit 9ca45e8

Please sign in to comment.