“Understand how clients use our products”
Introduction
ActiveViam has shown that its proprietary software Atoti is the leading technology to interactively analyse datasets from Gigabytes to Terabytes. Atoti can harness the largest machines of Cloud platforms and on-premise DataCenters to make all the CPUs work at a blazing speed on their Terabyte RAMs filled with client data.
Atoti is delivered to clients as a set of libraries. They are the building blocks to create their projects. This encompasses the setup of the data model: the tables and joins, the creation of new metrics and axes for analysis. It contains utility to load data into the project, and various ways of querying the data from the project.
Clients can build applications on their own. They often resort to buying consultancy from ActiveViam to build some advanced features, before integrating and maintaining it over the years, without intervention from ActiveViam teams.
In this context, ActiveViam is looking for ways to better understand their client projects, needs and blockers. Client teams being regularly busy, we look for non-intrusive automated and simple ways of inspecting projects, to gather as much information to enhance our products.
Expected Work
The goal of this internship is to prototype a system to do runtime analysis of client projects with respect to their usage of Atoti, the main product at ActiveViam. This is often referenced as telemetry on usage in many products. In the case of ActiveViam, as many client projects are very sensitive, an approach automatically sending information to an online service is not a viable option. The measurements must happen locally. They must be recorded as in a plain text format that can be inspected by clients, to ensure that no secrets are leaked.
In an initial phase, the intern will explore ways to instrument a project to record runtime information about the usages of ActiveViam tools. The result must be usable by clients without having to change the code, ideally without even having to recompile anything.
This phase will produce the specification for a report format, plain and readable.
In a second phase, the intern will study a second analysis approach based on Observability information. From operational traces produced by OpenTelemetry tooling, the program will extract ActiveViam services operations and record them.
In a third phase, the intern will investigate ways of computing runtime statistics about the method calls. The goal is to figure out how many times methods are called, how much time calls are taking, what calls are generating errors.
Finally, the intern will look for ways to control the amount of memory consumed by the inspection tool. Because Atoti is a product managing big datasets, we want to leave as much memory as possible to Atoti. With this in mind, we want our inspection tool to consume as little memory as possible. The intern will therefore introduce knobs to limit the CPU usage and memory consumption, with strategies to evict some data, possibly losing some information.
This prototype will be coded in Java, the core language of Atoti, the main product at ActiveViam.
To turn the prototype into a production grade component and make it part of the Atoti+ solution, this internship may open up the opportunity to be continued as a standard position in the R&D team.
About ActiveViam
ActiveViam provides business users with instant insight into large volumes of fast-moving data for timely and context-aware decision-making.
Founded in 2005, ActiveViam employs over 150 people in its five offices of New-York, London, Paris and Singapore. We expect sustained growth in 2022 and will continue hiring the best talents from the top schools.
