Orchestrating Image Preprocessing Workflows for Label Studio with Prefect.io

Michael: All right, thank you everyone for joining us today for another partner webinar. This one is very special. We’re fortunate to have Eugene Mandel, Head of Product at Superconductive, the makers of Great Expectations. If you're not familiar with it, you're about to be.

A couple of housekeeping items before we kick things off. First, if you'd like to submit questions during the webinar, please join our Label Studio Slack community. There's a Bitly link on your screen, and I’ll make sure it stays up throughout.

Today’s agenda: I’ll start with a few announcements, then we’ll hand it over to Eugene to talk about Great Expectations and how it relates to testing your data with Label Studio. We'll wrap up with a Q&A.

Quick announcements: We're launching a monthly newsletter, starting next week. It’ll include new blogs, tutorials, webinars, release info, and community content—all in one email, once a month. Subscribe at labelstudio.substack.com. First issue drops June 30.

Also, our next community webinar is coming soon. The last one was a fascinating discussion with a team working in the Indian municipal court system. We’re finalizing our next guest now.

We also have a new blog post: "10 Best Practices for NLP Labeling" by Sarah Moir and our CTO Nikolai. It’s live now, and you’ll see it in the announcements channel and newsletter.

All right, enough from me. Let’s kick it to Eugene.

Eugene: Thanks, Michael. Let me share my screen. Okay, here we go.

I’m Eugene Mandel, Head of Product at Superconductive. We’re the team behind Great Expectations. Just a bit of background—I’ve been in startups for a while, always working with data. My takeaway: better data usually beats better algorithms.

Before this, I worked on bots for customer support. The algorithms were off the shelf. The real work was building the data pipeline, especially the labeling system with human-in-the-loop components. That’s why I’m happy to be here today.

Great Expectations is an open-source project for testing and validating data pipelines. It was launched in 2018 by Abe Gong and James Campbell, who are now our CEO and CTO. It’s now the most popular open-source library in this space, with a large community and lots of contributors.

Michael: Can you explain what kind of data problems Great Expectations solves?

Eugene: Sure. There are two main types: pipeline risks and data risks.

Pipeline risks are technical—something breaks in your job or pipeline. Data risks are trickier—the pipeline works fine, but the data is bad or misunderstood.

Take drift, for example. Imagine you’re getting daily batches with an average price field. Over time, that value changes. If your model was trained once and never retrained, predictions get worse. That’s not "bad" data, but it’s not being used correctly.

Another example is outliers. If your training set includes a mansion with 50 bathrooms, it can skew results. The point isn’t to say data is good or bad—it’s about whether it’s fit for purpose.

We also see pipeline outages. Let’s say your log generator dies for six hours. Downstream, you might assume nothing happened—but actually, you have missing data.

Michael: So how does Great Expectations help with that?

Eugene: It uses "expectations"—declarative statements about what you expect from your data.

Example: "I expect values in this column to be between 1 and 6."

You can define expectations in code, YAML, or even natural language. They run against your data and return validation results—pass or fail. If they fail, you get detailed feedback.

There are many built-in expectations: column exists, values aren’t null, values are unique, means fall within a range, distributions match, and so on. Contributors also create domain-specific expectations, like valid bank routing numbers or drug codes.

Michael: Where do these expectations come from?

Eugene: Two places: domain expertise and profiling. Profilers can analyze historical data and generate draft expectations. Then a human reviews and edits them. That combination works well.

Michael: And how do people see the results?

Eugene: Great Expectations generates "Data Docs"—HTML reports that visualize validation results. If a test fails, the doc shows why and where. This makes collaboration easier. It beats long email threads trying to describe a problem.

We often say: your tests are your docs, and your docs are your tests. Documentation becomes a byproduct of testing, so it stays current.

Michael: Is this only for Python engineers?

Eugene: Right now, yes. Most users work in Jupyter Notebooks. You need to be comfortable editing YAML or writing basic Python. But we’re building a SaaS version with a GUI, permissions, and collaboration tools for non-coders. Join our Slack to get updates on that.

Michael: And what about using this for ML models in production? Can you validate inputs and outputs?

Eugene: Yes. You can validate both. If a model sees an outlier it’s never trained on, it shouldn’t just make something up. Great Expectations can filter those out or flag them. You can also validate predictions—like making sure the price prediction isn’t negative.

Michael: Awesome. Anything else to add?

Eugene: Check us out on GitHub, read the docs, and join our Slack. The community is fantastic, and we’re always looking for contributors.

Michael: Thanks again, Eugene. That was fascinating. And thank you to everyone who joined. See you next time.