.jpeg)
While there are thousands of SaaS products out there and a lot of them have absolute substitutes, there is not a single team of product builders that's absolutely the same, hunting for the same metrics, and trying to answer the same business queries. With that in mind, it doesn’t matter how many product analytics tools there are in the world, we are always looking to solve one or the other problem statement.
From Data Dilemmas to Actionable Insights, there are still many problems that are persistently faced by PMs and analysts then it comes to analyzing their product’s usage, despite the rapid proliferation of new analytics tools. We explore some of these problems.
Data compilation is just the first step, making sense of the collected data is what matters the most. While there are analytics platforms (albeit not too many), that are phenomenal when it comes to the collection of data but doesn’t help much in making sense of it. Probably because of problems like an unintuitive UI/UX for dashboard setup, irrelevant integrations, and data tracking/filtering options.
Modern day LLMs are a breakthrough for this problem, because the user can engineer the prompt as per their requirements and generate the user data in a way that’s of most value to them and their role within the organization. Moreover, they can do this using natural language.
The significance of being able set up mechanisms to handle your data in English vs code cannot be understated:
While to seasoned analysts it may seem like a mere nice to have, the impact of such accessibility is that it drastically decreases how easily others on a team can leverage data. For example, your. marketing team may well benefit greatly from knowing and using product data better. However, to do so, they'd likely have to go through someone on the team well acquainted with usually complex analytics tools. Using Natural Language for the same solves this problem.
Similar to and intertwined with the aforementioned factor, the idea of creating another dashboard or tracking a new event/metric might come from the product team or the business analysts or the CS teams, but the tagging and designing part of the dashboard creation is done by engineers and PMs.This tends to result in a dashboard that just might be easy to comprehend for the product team, but not so much for the CS teams. As a result, this creates a massive gulf in communication and alignment within the teams.
What we are also looking at here is not just how the dashboards can be complex, but also how the same data from the same source can be perceived differently by different teams within an organization.
With generative AI exponentially advancing, it shall become even easier for all the teams to perceive the data consonantly since it’ll be portrayed in a rather plain language.
This leads to dashboards and charts that are inaccurate. With most product analytics platforms needing you to painstakingly tag events, it is even more important to have appropriate data governance in place to ensure there are no discrepancies when the dashboard is finally live.
A lot of the success comes down to your organization's implementation of the front-end events. Some time spent upfront on event strategy and implementation is essential to really get the most out of most product analytics tools. Data governance is key here.
Unless you are 100% certain it is set up properly, the data does not and frankly, should not, engender trust.
A viable solution for this problem would be to opt for an autotracking enabled analytics solution that tracks data intelligently, understanding what all the events actually are.
Especially for products that are just starting out, not being able to distinguish 'real' usage from your own team members engaging with the product for developmental purposes is quite an issue.
For example, if your product has just launched and there are 100 engineers working on it daily, the data derived will be highly inaccurate, with false events sometimes even going into the thousands.
Now, despite there being several solutions to circumvent this problem, they don't come without problems of their own:
DET, now available with many analytics platforms,tends to be quite substandard, and as a result, provides paltry value.
For example, the auto-tracking of 15-20 events by default definitely saves time, but it still requires the PMs and engineers to set up tagging for other, more crucial events. Furthermore, if you already have a dashboard setup then it doesn’t really affect you much.
Given how useful DET can really be, removing the downsides that currently come with it would be a game-changer.
Using a purpose built, fine-tuned model, this is now possible.
Now, we dive deeper into the experiences and insights shared with us by industry professionals through one-on-one interviews. From dashboard creation to data accuracy, let's explore the key challenges faced by PMs and analysts in their day to days workflows.
Creating effective and informative dashboards is a crucial aspect of product analytics. However, our interviews revealed various challenges related to dashboard creation, updates, and distribution. The time and effort required to set up events, track them, and design intuitive dashboards can be significant.
Moreover, keeping dashboards up-to-date can be a demanding task, requiring continuous monitoring and updates. Additionally, sharing dashboards with teammates and ensuring they are as accessible to all teammates is near impossible, given the varying degrees of expertise across verticals in a company. As of yet, there's not really been a solution for this.
However, by being able to make dashboards almost an order of magnitude more quickly (that too using plaintext) enabels people across your team to do so. This enables dashboards to be created for hyper specific levels of expertise and use cases.
Achieving data consistency and accuracy is a recurring challenge faced in product analytics. Ensuring that data is collected accurately across different tools and platforms can be complex, especially when inconsistencies arise. Factors such as ad blockers and data discrepancies across various analytics tools can hinder the accuracy of insights derived from the data. Resolving these issues often requires meticulous troubleshooting and rectification efforts.
As of yet, there’s been no viable means to accomplish this. However, training purpose-built LLMs to disregard anomalies in the data collected are now tenable. Such a model would be capable of removing any and all data points that were adulterated because of ad blockers, sync issues, technical glitches, etc.
Perhaps the most crucial challenge. Extracting valuable insights from the data and using them for effective decision-making is a primary goal of product analytics. Our interviews revealed that industry professionals focus on metrics such as user behavior, segmentation, and funnel analysis to gain actionable insights. Ctreating the right type of dashboard and maintaining it with every new feature update can be a tedious, and more importantly, error prone task.
An AI model in such a case can be a breakthrough by being able to suggest the types of charts that should be created to monitor a particular set of business questions.
As organizations increasingly rely on data-driven strategies, the challenges encountered in product analytics become more significant. From creating comprehensive dashboards to ensuring data accuracy, industry professionals face a variety of obstacles on their path to deriving actionable insights. The experiences shared by the interviewees emphasized the importance of continuous tooling evolution, learning, and a good understanding of the organization's specific needs. By addressing these challenges head-on, businesses can unlock the true potential of product analytics and make data-informed decisions that drive success in today's competitive landscape.
At Crunch, we're solving every problem spoken about here by weaving modern AI with analytics.
Sign up for our waitlist here!
