The City of San Diego has recently expanded its green waste recycling program to include weekly organic waste recycling collection for all households currently eligible for collection services. This is good news as it'll enable the City to get closer to its goal of reducing waste being put into landfills. The City’s goal is to reduce waste going into its landfills almost entirely by the year 2040. The Environmental Services Department (ESD) wanted to use smart technologies to efficiently manage the new green bins and greenery collection. The technology employed includes three automated business systems: 1) work management, 2) GIS, and 3) fleet management.
The new weekly organic waste recycling collection includes curbside pickup of food scraps, food-soiled paper, and yard waste. Some residents have already started receiving their new green bins, and all neighborhoods should have curbside greenery pickup over the next several months. Cities can substantially increase the life of their landfills by recycling green waste. The city has estimated that over three cubic yards of landfill space are saved for every ton of greens recycled. With numbers like this, it is obvious why, in 2022, the City of San Diego began a significant expansion of its green waste management program. The Environmental Services Department (ESD) in San Diego ensures residents have a clean and safe environment by pursuing waste management strategies that emphasize waste reduction, recycling, composting, and environmentally sound landfill management.
ESD already had GIS to manage waste collection routes for refuse, recycling, and green waste and wanted to leverage that investment for the expanded organics pickup program. The GIS software ESD uses for collection is based on Esri’s ArcGIS Platform. The department also had work management software, Salesforce, to manage the weekly bin pickup schedules. The third, and new system added to the mix was chosen to provide fleet management capabilities. ESD chose Routeware as the additional software installed on in-cab units, which gathers data from smart sensors and cameras mounted on the waste collection trucks.
The City of San Diego needed an integrated interface to pull together data from these three specialized systems to reduce staff manual labor and increase efficiency. Quartic Solutions, a San Diego-based woman-owned GIS services company, was brought in to assist with the integration. Quartic’s job was implementing an automated process to keep data current between these systems. As Salesforce, ArcGIS, and Routeware are all modern technologies, they all utilize web service technology for communication. Quartic created the web service components (ArcGIS feature and map services) that bring Salesforce and Routeware data together.
Using web services, the team could design and construct a smart system architecture that would meet all drivers, supervisors, and office staff requirements. The team redesigned the previous Extract Transform Load (ETL) job to handle the complex table structures needed to keep the location site data, the bin container data, the route data, and the fleet data organized and in sync. They also created a REST endpoint for Routeware, so truck drivers could update the routing information.
The batch data synchronization process begins with Salesforce updating records in multiple GIS tables via an ArcGIS feature service. GIS synthesizes information, performs geoprocessing at the container level, the smallest unit among multiple tables, and brings in additional information from various reference layers. The intermediate product includes the truck routing information sent to Routeware via a feature service. Once Routeware reads data from GIS, routes are displayed in RCC for the drivers to view and update.
After drivers pick up all the bins and make route attribute changes, the updated route information returns to the GIS database via the same feature service. GIS performs another batch of geoprocessing to update individual polygons representing routes per bin type. Then, the container-level data is converted back to the site-level data, and the updates are sent to Salesforce. Salesforce pulls the GIS delta via a feature service, and the department accesses the dataset via either the map service or Enterprise database connections.
Integrating complex systems can be tricky. The new system had to manage much larger datasets than in the past. ETL jobs are scheduled to run at night when system usage is lowest to keep data in sync between the three systems. The Salesforce, GIS, and Routeware batch jobs are also scheduled to run in a sequential order at specific times, nightly. If any one of the batch jobs takes too long to complete, it'll likely have an impact on the subsequent job run. To improve performance, the team employed a solution called multiprocessing. This technique allows the GIS software to process large amounts of data within limited server resources. The ETL job run time was significantly reduced using the GIS multiprocessing technique. Then, to improve performance even more, the GIS was configured to take advantage of database structures, called database views. These database views speed up the processing by having the modified (aka delta) records in Salesforce and Routeware indexed and in memory, greatly shortening the data-pulling time.
The Esri ArcGIS technology used in the project was ArcGIS Pro, ArcGIS Enterprise, and ArcGIS Online/Portal.
The new system allows ESD access to the functionality they need to effectively manage the organic waste recycling program. Now, drivers can use in-cab tablets loaded with quality spatial data. In-office staff can also be more productive using an integrated Salesforce, GIS, and Routeware system. The benefits are anticipated to increase staff productivity, reliability, and response times to the public. The project was planned and scheduled to roll out in three phases. The first two phases are complete, and the final one is estimated to begin by the end of the year.