Loop internship highlights: Pierce Maloney.

When I first arrived for my summer internship at the Loop office in San Francisco, I greeted and chatted with Shaosu. As he set me up at my new desk, Shaosu told me that we would be moving to a bigger office soon. At the moment, I didn’t quite see the need to move, but I soon learned. One by one, as they arrived, I met my coworkers as the office started to fill. By mid-morning, it was nearly full, and there was a great, productive energy present.

All new engineers push code on their first day, signifying the company's agile and iterative approach. This was further emphasized in our all-hands that morning when Matt relayed to the team how impressed a customer was with how quickly we implemented multiple requested features.

The internship project

After the all-hands, I sat down with Kevin and Evan, two incredibly talented senior ML engineers who were my mentors for the summer. They walked me through our development workflow. I was immediately struck by how well-designed Loop’s workflow was for a fully automated future – the data ingress is built upon a system that generates a collection of relevant “tasks” used to continuously upgrade our ML models – but perhaps the most striking piece was how relevant our work was. Not only was Loop positioned at the cutting edge of the field, we were pushing it forward, combining the newest ideas and creating novel model architectures for the tasks at hand.

Then, Kevin and Evan introduced the project they had in mind for me that summer. I was to build and implement a model that extracts the relevant fuel surcharge for a given week from a rate agreement document (the image is an example of a 72 page public Fedex contract). To someone unfamiliar with freight logistics, this doesn’t mean much, but those familiar know this task well. I did not know the best way to go about this, so I asked them. They looked at each other and said, “Let’s figure it out.” We spent the next half hour brainstorming the possible approaches and decided on the one we believed to be best. From that point forward, it was clear that my job for the summer would consist of a series of creative solutions to new problems, which I always hoped for but didn’t expect as an intern.

Something that underpinned the whole experience – as Evan drew on the whiteboard and Kevin jumped in to add various pieces – was the feeling that we were doing something special. This was a feeling that was difficult to distinguish from initial nervousness or excitement for the months ahead, but I distinctly remember six weeks into my internship when that sentiment had not wavered. I turned to Evan and asked him about it. He smiled and nodded, and told me I wasn’t crazy; Loop was a special place to work.

The problem I worked on was a technically difficult one: building a multimodal language model to extract fuel surcharge tables from multi-page (often 100+ page) documents of all different formats. There was limited, mixed-quality data, and many out-of-the-box models struggled with numerical reasoning, which the task required to a degree. We approached these thoughtfully yet aggressively, and each week my methods would look different from the last. But the role Kevin and Evan gave me was more than just model design. From data collection and cleaning, model training, to deployment and full-stack integration into the product, I was trusted to touch many aspects of what was being built at Loop. By the end of my stint, the model I created is set to save Loop many man-hours annually, particularly as Loop scales – a testament to its real-world impact.

Results and presentation

Perhaps the most inspiring moment for me came after meeting with the heads of Customer Success, who oversee the auditing at Loop, and presenting them with the product. They expressed a desire to see evidence of the model’s reasoning, which practitioners know well as the problem of explainability. After this feedback, in less than two days I turned around and presented a new model using chain-of-thought reasoning to incorporate this feedback into my models. Each of the two rows in the below image were a separate model inference call that chained together to provide explainability, so we could understand how the model got to the final answer. By the end, the model had achieved over 99% accuracy.

After implementing this, as well as updating it with new data periodically, in my final few weeks I got to explore novel model architectures to solve other, relevant problems. A strong theme across every project I worked on during my internship was growth; every day I could work faster, write better code, and contribute more than the last.

So far, my experience at Loop only lasted a summer, but the lessons here are perpetual. It was the warmth that welcomed me in and encouraged collaboration. The growth mindset promotes learning and keeps Loop pushing the boundaries of what’s possible. The speed of iteration allows for rapid exploration of the intersection of what’s possible and what’s best. And finally, trust is what allows for deep project ownership and maximizing each person’s individual contributions. I had high expectations for my internship at Loop, and they were exceeded in all regards.