Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce GLS Pilot blogs. #295

Merged
merged 5 commits into from
Jan 4, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions blog/2025-01-05-antonius/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
slug: creating-driver-areas-for-parcel-delivery
title: "Creating driver areas for parcel delivery"
authors: [antonius]
tags: [introduction]
unlisted: true
image: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/social_jennings.png
hide_table_of_contents: false
keywords: [introduction]
---

In the parcel delivery business, geographical evaluations are crucial. Do delivery drivers visit the same regions each day, letting them know their areas intimately? Or is there a high volatility of the areas? And of course, do multiple drivers go to the same region, leaving potential for improvement?
Those are the questions Antonius is working on at GLS Studio, an innovation lab by GLS (General Logistics Systems), an international parcel delivery service provider.
While the planned areas of driver tours can be rather well-defined, an evaluation of the areas actually served by drivers is equally important and might not be as easy. It can be used for guiding delivery drivers to become more efficient while also enabling managers to identify planned areas which are suboptimal due to limiting factors like rivers or big highways which have not been considered in the planning.
In this blog post, Antonius highlights the use of Fused to display the areas of a single day as well as aggregate multiple days with the purpose of creating stable driver areas.


{/* truncate */}


## The challenge

Creating areas out of single geospatial data points is challenging. A naïve approach would be to use convex polygons. This causes multiple issues: a single data point can have a high influence on the shape and area of the polygon, a tour consisting of multiple not connected sub-areas is hard to detect and correctly display, and just as important this limits the area to what is covered by historic data, so when intending to use areas for the future, there remain gaps which have not been covered historically. Lastly, a calculation using polygons is computationally expensive, hindering ad-hoc changes to the selected time span or the calculation parameters.

A further challenge is that building new and sometimes experimental features means that the database is often not optimized for the use case. Data needs to be joined between multiple tables and even between multiple data sources. Therefore, fetching all data can take a considerable amount of time, highly limiting the usefulness of having an experimental feature to play around with.


## The solution
The solution to the first problem came by using H3 cells instead of polygons. By assigning cells to the drivers based on their historic deliveries to the cell, driver areas are an automatic result. Using H3 cells across different resolutions also allows to represent the differences between urban and rural areas. While there exists one ‘base resolution’ to ensure non-overlapping and complete areas, the logical hierarchy among H3 cells can be used to calculate on lower resolutions for rural areas, speeding up the computation as well as ensuring a broader coverage of those areas beyond the historical data points. On the other hand, disputed H3 cells can be broken down to a higher resolution and assigned to different drivers or, when the ‘base resolution’ has been reached, assigned to the driver delivering most parcels to the cell. As H3 cells have clearly defined neighborhoods, areas can easily be extended beyond their historical limits when desired, covering the empty space around to also catch tomorrow's parcel which is just beyond the historical area boundaries.

Regarding the latency of querying and calculations, Fused UDFs came in handy. When a user looks at an area for a day, they are probably interested in the same area on some of the previous and following days as well, right? So why not pre-calculate that already. Using Fused, it is simply a matter of fire-and-forget to trigger the UDF with parameters for some previous and following days which are then already running to cache. So when the user is now checking out the day before, the data will already be there. And more in general, when it is possible to limit the number of parameter combinations in an experimental feature to a manageable amount, this fire-and-have-it-cached approach is not limited to caching data from previous and following days, but can also be used for a range of other cases.


## Conclusion
When developing new features which are not yet supported by the current data infrastructure, Fused UDFs enabled us to easily test things without having to change the underlying infrastructure in advance. The UDFs are easily shareable and adjustable, allowing testing by multiple people without having to run code locally while automatically making sure everyone is using the same code which is hosted in the UDF. And now, when we have verified the feasibility, the first feature created using Fused is about to be integrated into our product.
45 changes: 45 additions & 0 deletions blog/2025-01-05-kyle/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
slug: how-fused-became-the-needed-boost
title: "How Fused became the needed boost in the Fiber race"
authors: [kyle,nelina]
tags: [introduction]
unlisted: true
image: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/social_jennings.png
hide_table_of_contents: false
keywords: [introduction]
---

Pilot Fiber is a commercial Internet Service Provider primarily serving customers in New York City. Our primary value proposition in competing with national-scale ISPs is our commitment to customer service– we actually do care when any type of incident interrupts your service and will immediately jump into action to address the cause.


{/* truncate */}

### Introduction



Almost all fiber optic cable in Manhattan runs through a shared manhole-and-duct system, meaning that road construction or work by other providers in a manhole has the potential to damage the equipment of multiple providers. Because of this, Pilot uses an active fiber monitoring system across our network; devices located in our data centers are constantly shooting light down the fibers in our network and looking for any anomalies in comparison to a “fingerprint” stored when that fiber was initially installed to a building.

When that system registers an anomaly, it will immediately fire an alert giving a fiber route and distance to the potential problem (i.e. “There is a light loss on the fiber running to 1234 5th Ave at a distance of 2.351 kilometers from the data center.”). When this happens, our teams need to be moving within minutes to determine the exact location of the issue and, if necessary, get crews headed to the site to begin repairs.

## Body

Historically, this process would require the attention of an Outside Plant engineer with access to specialized software and network knowledge, regardless of the time of day or day of the week. With Fused we are working to make this information accessible to more support team members and automate the creation of the reports field crews need to address the issue when they arrive on site.
Using Fused as a back-end glue layer, we have built a web app that allows a less technical user to select a route and distance and map where the system says the fault occurred, view nearby network infrastructure, and automatically generate the necessary field reports based on that nearby infrastructure.
The workflow requires a series of calls to UDFs that act as intermediaries to a Postgres/PostGIS layer, which in turn is sourcing data from other internal sources. This structure allows us to easily keep the business logic organized at the UDF layer while limiting the scope of data access and security via Postgres and internal processes maintaining the sync.
The basic process is seen below:


When the user selects a route and a distance to process, two separate processes are initiated. One retrieves the selected route to load onto the Mapbox-based map within the app, while the second kicks off a processing chain. This chain is utilizing UDFs that both assist in isolating the location of the fault and relevant nearby infrastructure as well as adding elements to the map display to assist the user in visualizing what may be occurring.

UDF - returning the route itself that will show up on the map.
UDF get fault point php
UDF - putting the red dot on the map.
UDF - generate the splice report.

Providing non-technical users a way to have access to the data in Postgres this democratizes where you could have theoretically anybody at the company gather the information to get the splice team headed in the right direction as fast as possible and know what they're looking for and what they're looking at when they get there. All this person would need to know is what the distance was and what building was affected. By using these different UDFs and Postgress the tool will figure out the point and if relevant, it will generate the documentation that they need to give to the field crew. This improves the turnaround time we dispatch somebody to the site.
I could do a lot of this just with Python but there were no visual maps to work with in these scenarios. Fused works so easily with the map and the calculations also give a lot of context to the person who is running it. So even our more technical personnel can see and make a judgment call about what they think is actually happening or not happening here.

## Wrap up

...
23 changes: 23 additions & 0 deletions blog/authors.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,3 +159,26 @@ jacob:
title: Senior ML Engineer @ VIDA
url: https://www.linkedin.com/in/jacobbieker/
image_url: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/authors/jacob.jpeg

kyle:
name: Kyle Pittman
title: Sr. Director, Systems and Network Strategy @ Pilot Fiber
url: https://www.linkedin.com/in/kwpittman/
image_url: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/authors/kyle.jpeg

nelina:
name: Nelina Huang
title: Data Associate II @ Pilot Fiber
url: https://www.linkedin.com/in/nelinahuang/
image_url: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/authors/nelina.jpeg

antonius:
name: Antonius Moosdorf
title: Data Scientist @ GLS
image_url: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/authors/antonius.jpeg

maribel:
name: Maribel Hernandez
title: Professor @ CINVESTAV
url: https://www.linkedin.com/in/maribel-hern%C3%A1ndez-758b24134/
image_url: https://fused-magic.s3.us-west-2.amazonaws.com/blog-assets/authors/maribel.jpeg
Loading