Tech Blog Menu

At Coveo, we deal with an enormous amount of data on a daily basis. With data growth, our data platform has also grown from a single team a few years back, to more than 3 teams and 20 employees. With this growth, we also gave ourselves the mission of democratizing data across our organization and allowing more and more external teams to access and experiment with the data we capture.

The challenge we rapidly faced was that we had to offer more and more support to these external teams on how to automate some of these applications and scripts they were developing over the data. Most of these stakeholders are often really proficient with SQL and Python, but have less knowledge and experience with CI/CD, infrastructure, and monitoring.

To solve this problem, we started looking at some solutions that would allow these teams and individuals to easily deploy and run these different workloads in production, without having to develop an in-house solution that would require a lot of engineering time and maintenance.

After investigating multiple solutions to solve this problem, a clear winner stood out for us: Prefect.

Next summer, I’m going to drink a beer under a tree.

Right now you probably have an image of me, under a tree, with a beer: you understood my intent. What’s interesting is that the details of this image will probably be significantly different for each person that is reading. This is because I did not talk about the implementation; beer and tree are abstractions. They efficiently convey a general idea without getting lost in the details. Efficiently is the key word here; imagine if I had to describe the detailed implementation of a tree or a beer each time I want to talk about them! 😰

Since abstractions are a key part of the way humans communicate, they are also a key part of software engineering. Indeed, since software engineering is about programming over time, it is in part about using source code to communicate ideas to other developers. This won’t come as a surprise to most of you; we, developers, know that abstractions are a key part of our job. That being said, it’s not always clear how abstractions link to our day-to-day programming and to our leading practices. Single Responsibility Principle (SRP), Design Patterns, Don’t Repeat Yourself (DRY), and test readability are all somehow related to abstractions. This post explores these connections and shows how understanding them helps to make better decisions about our programs.

A few years ago, Coveo wanted to offer the possibility to store customer data outside the United States. Early in the project, it was decided that we would only have one user directory. If our users had to manage organizations created in different regions, we wanted them to avoid having to log in each region individually. This meant that we would have only one database to store users. This blog post will explain how we achieved a central user directory while avoiding big latency issues.

Through the years, I’ve heard many different opinions about code review, ranging from “publishing code to the main branch with extra annoying steps” to “I learn so much from it” stepping through “it’s pair programming with delay” and “I can’t push to prod unless someone looks at my stuff”. Being a big fan of code review, I’d like to share how I make the most of it and how I can learn from people who are awesome at it and ideally help you get to a place where you can provide a lot of value with it.

Coveo has experienced great growth over the last few years, by bringing in new clients, deploying in new regions, integrating new technologies, etc. But the infrastructure on which our offering sits must follow the same trend. This explosion in data and event volumes demands that companies find scalable solutions to match their ambitions. It’s a bit more complicated than simply throwing buzzwords like incantations, so I’ll help you dive a bit into this world.

Coveo’s Query Suggest model provides highly relevant and personalized suggestions as users type. In this blog post, I will explain how Query Suggest works in the back end, and how it uses mutli-threading to provide results at high speed.

Do you want to have better and faster search results in your Coveo-powered catalog search pages? You can do it by creating an indexing pipeline extension (IPE) that identifies and stores all the variations of your partial SKU values.

You feel like you do a decent job at creating new user intefaces or modifying existing ones, but writing tests to cover your changes takes you more time than to write the actual source code. You have experience writing tests for clear cut units of code like functions or classes, but it seems that testing UI is just too different. Having lost all intuition about what needs to be tested, what needs to be mocked, how to split your test cases, or even how to write a simple assertion, you feel lost and unproductive.

If you can relate to any of the above, this article is for you. This post aims to provide a UI testing “booster pack”, all batteries included, that will get you rolling in no time. Having the right mindset and using the right tools will make you almost look forward to writing tests for UIs. Not only because your tests will grant you confidence and improve the quality of the code you produce, but also quite frankly because it will become an enjoyable thing to do.

Making search engines consistently fast is extremely hard. They have tons of interconnected components, and a minor degradation in one part of the system can easily spiral out of control and become a critical performance issue. Caches in particular are especially crucial to get right, because they can cause a great variety of problems: sub-optimal memory usage, query performance degradations, excessive contention in hot paths, etc. In this post, we’ll cover the current state of caches in the Coveo index, as well as the design and implementation of a new caching library able to solve our issues.

At Coveo, we use Prometheus 2 for collecting all of our monitoring metrics. Prometheus is known for being able to handle millions of time series with only a few resources. So when our pod was hitting its 30Gi memory limit, we decided to dive into it to understand how memory is allocated, and get to the root of the issue.