monthlydumpsparquet1 monthhourly agg.by stationCitybikesscraping the world forbike sharing dataso you don't have toIntro - 2009ackCityBikes: the new stuffpublishermetricsschedulerzmq socketsubscribersubscribersubscriberthread poolpybikestaskhyperThe android app that never wasa normal pipelineworkerspybikespybikespybikespybikespybikes...multiagent systempublisherpybikespybikespybikespybikespybikes...REST APIreal time subscribersqlite~ 50 MbREST APIstats subscriber(4 weeks)sqlite~ 40 Gbpeople fromthe internethot stats subscriber(1 day)sqlite~ 1.8 Gbstoragecold db(DuckDB)monthlydumpsparquetmultiagent systempublisherpybikespybikespybikespybikespybikes...30% gbfs (230)3% licensed (21)17% sourced (132)gbfs-ish (147) 32%other (147) 18%CityBikesAPIGBFS LayerHistorical bike share dataGBFSworstGBFS+pybikes