Uncovering Hidden Opportunities to Optimise Costs with Knowledge Graphs

What We Set Out to Solve

Understanding and controlling costs is critical for any organization running large-scale compute systems. But with a dataset that spans performance metrics, configurations, and usage patterns, how do you pinpoint the key factors driving your costs?

Using Conode, we explored a sample of a 50GB dataset to answer this question: What compute or server configurations could help reduce costs while maintaining optimal performance?

---

How We Approached the Problem

1. Exploring the Dataset

The first step was to upload a subset of the data into Conode. This allowed us to get a high-level view of the data structure and features we were analyzing, ranging from memory usage to performance and beyond.

👉Loading the dataset, visually exploring features, and using Conode AI to tell us what it contains.

---

2. Identifying Patterns with PCA

To make sense of the complex relationships in the data, we applied Principal Component Analysis (PCA). This dimensionality reduction method grouped related features close together on the graph, showing clusters that revealed underlying correlations. For example, features related to memory usage naturally grouped together, while others tied to performance metrics formed separate clusters.

Additionally, we used the Conode AI agent to generate representative groups of these clusters.

👉 PCA visualization with feature clustering.

---

3. Removing Noise and Outliers

Outliers and redundant features can mislead the analysis. To ensure accuracy:

• Features explicitly related to “price” were excluded to prevent them from skewing the prediction of the primary cost-related target.

• We first look into features that have high correlation with our target variable (price projection total) and removed anomalies from their scatter plots.

👉 Cleaning the dataset by removing confounds and outliers.

---

4. Predicting Cost-Driving Features

Next, we used a prediction model to identify the features that most heavily influence the target variable: total projected price.

• The model pinpointed key contributors from the dataset.

• These insights revealed which configurations directly impacted costs, highlighting where optimizations could yield the greatest savings.

👉 Prediction model in action, revealing top cost-driving features.

---

Key Takeaways

This exploration demonstrated the power of combining graph analytics with AI:

• PCA provided a transparent overview of data relationships, letting us intuitively identify critical features.

• Predictive modeling highlighted specific areas for optimization, translating complexity into actionable steps.

• By eliminating noise and honing in on influential factors, organizations can make informed decisions to lower costs without compromising performance.

Conode’s visual-first approach transformed the dataset into an interactive, analyzable framework, akin to the intuitive interfaces seen in Minority Report—but designed for your data.

Whether you're exploring compute configurations, unifying disparate datasets, or seeking insights at the intersection of performance and cost, knowledge graphs open doors to better decision-making at scale.

Get In Touch