“AI” is an umbrella term with quite a few different meanings, depending on who you ask. For some, it sounds like the coolest thing ever; for others, it signals the end of humanity. Here at Memory, we use the term to refer to a specific branch of AI called “Machine Learning”. In this article, we’ll break down exactly what that means and how it allows Timely to intelligently draft time entries for you.

Machine learning

First things first, let’s define the form of AI we use for Timely. “Machine Learning” (ML) is a set of methods and practices that aim to solve problems for which solutions are either a priori unknown or unachievable. It’s not about telling the AI to use a known process — it’s about getting it to work one out for you.

Even within ML, there are lots of different ways to do this, but we use the branch called “supervised learning”. You can use supervised learning when you know the inputs and outputs of a system (the data), but can't efficiently conceive the transformation(s) which get you from A to B.

ML and Timely

With a ton of time data and the ability to show the model what project each piece belongs to, Timely provides an ideal problem for ML to solve. As a user, you just need to:

  1. Install the Memory app to create a digital memory of all your work;
  2. Log tracked events (or “memories”) to the right project.

In doing so, you create a relationship (a “pair”) between each memory and your projects, as follows:

  • the work memory is the input of our system;
  • the project is the output.

To code that relationship, you could hire an army of human analysts to study such pairings all day and perhaps they would be able to pick an appropriate project when presented with a new memory to log… But that would be really expensive, inefficient and a massive violation of your privacy, so instead you can just use ML to do it!

ML paradigms can be organized along many dimensions. In Timely, the unidirectional relationship between a memory and a project is modeled with a classifier. Classification makes up a large number of ML applications, but there are many other forms of ML too — including unsupervised learning, reinforcement learning and generative modeling.

Are you smarter than ML?

Ok, ok — it sounds quite involved, but how smart exactly are these classifications? Many people are skeptical of ML performance, believing it to be inferior to human intelligence. While AI is perhaps not as smart as humans, it can still be extremely powerful.

Still not convinced? Then take a look at the following classification examples; can you beat the machine by finding the relationship for each problem? (Answers are at the end of this article 😉 ).

Problem 1

1, 2

2, 3

4, 5

102, 103

Problem 2

2, 3

4, 6

12, 18

20, 30

Problem 3

10, 6

1, 32

1, 76

2, 98

3, 120

5, 164

10, 274


Before going under the hood with Memory AI, let’s explain a few concepts. We’ll restrict these to focus on classification only, which in the computer science hierarchy is a sub-category of supervised learning (which is a sub-category of ML, which itself is a sub-category of AI) — 🤓 phew!

∑ Model

A model contains ordered mathematical operations to apply to an input. The result of the last operation is the output of the model. In Timely, a model takes the features extracted from a memory and outputs the project to which it likely belongs. (Skip to “Walkthrough” below to learn how Timely does this).

✨ Features

Since a model contains mathematical operations and mathematics works with numbers, there can be a mismatch between the composition of a memory and what a model can accept. This is often because your work memories contain attributes made up of characters and timestamps. To get around this, your work memories (inputs) go through a feature extraction process that converts the raw data into data points or sets of numbers. Each extracted component of a data point is called a feature, and each data point is made of a set of features. We use “data”, “data points” and “features” interchangeably throughout the rest of this article, since they are all acceptable inputs to an ML model.

🏷️ Labels

Just like raw input data and features, raw output data needs to be converted from its original form into numbers. The converted set of numbers is called a label.

🧠 Neural network classifier

In the brain, excited neurons transmit tiny amounts of electric current to other linked neurons, and the cascading stimuli forms a chain which we recognize as creating memories or triggering muscle movements. Neuronal connections are constantly changing with sensory stimulation and experience. Science, currently, can't fully explain the “hows” and “whys” of the working brain, as it’s difficult to observe what happens when brains are given inputs.

Why are we telling you all this? Because this human neurological structure has directly influenced neural networks used in classification. Neural networks in supervised learning are essentially a highly versatile type of model inspired by the structure of the human brain. They leverage two properties of the brain:

  1. numerous neuron connections performing complicated operations
  2. constant changes in connection configurations

The intensity of the electric currents propagated through the brain are values in a neural network that can be measured in terminal neurons. The weight of the connections between neurons can be altered; zero-weights are equivalent to the absence of connection.


Now you’re completely clued up on ML’s classification algorithms at a high level, let’s take a look at how Memory AI puts these to use on a daily basis.

Memory AI performs two main tasks for you to create draft timesheet entries:

  1. Training, where Memory AI observes you linking memories to projects, learning the relationship between them to construct a model.
  2. Inference, where that model is then used to make predictions about which projects new memories should be logged to.

How Memory AI trains

In case you’re interested in the technical ins and outs of that whole process, here’s roughly what it looks like, end to end:

  1. Feed the network: Memory AI lives in data clouds which contain clusters of memories. Memory AI collects memories and then extracts features from them, which form the training dataset. Features are fed to the model, which initially is made up of random operations. As such, generated outputs seem to be random as well. Here the model makes a prediction for each memory, to the best of its current knowledge, about which memories belong to which projects.
  2. Measure error: In order to improve the model, Memory AI needs to figure out when it’s right, when it’s wrong and, in the latter case, what went wrong. The information is called the error or, more technically, the “loss”. This tells the algorithm what changes are likely to improve the model prediction accuracy.
  3. Update weights: The most critical and foundational idea of ML is that models are updatable. You feed inputs into a model, collect the outputs and measure the difference between the real and expected outputs (the “error”). Then, unless the model was perfectly accurate, its components should be updated and modified to make improvements, with the ultimate goal of making better predictions.
  4. Rinse and repeat: Once we’ve adjusted the weights, we give the updated model the same data from step 1 all over again and collect its outputs. Because the model's inner operations have been modified, we get different outputs to the ones collected in the previous cycle. New outputs means new measured error (which is hopefully lower, but it’s not always the case!). This process then gives us new updates to apply and train over again.
  5. Stop: Obviously, training has to stop at some point. That’s why you have stopping conditions to help you decide when to exit the training loop (steps 1 to 4). It could be as simple as counting the number of cycles, or as elaborate as gathering performance metrics on holdout datasets or derivatives of the loss value series. (If you didn’t get the last part of that sentence, don’t worry; it’s just Memory AI humble bragging!). 😝

How Memory AI infers

Once training is finished, Memory AI moves on to its next task: inference. The inference phase is like a truncated version of the training phase. It has only two steps, and it assumes that a model has been previously trained. Here’s what it involves:

  1. Feed the network: Memory AI fetches a memory (like an email you just sent or a file you’ve been working on), extracts features, and feeds them to the model's first operation.
  2. Get results: Memory AI's models provide two values as output for each memory they get: a project prediction and its likelihood. The likelihood is code for how confident Memory AI is when claiming a memory should be logged to a specific project. If the likelihood is too low, Memory AI should refrain from sharing its opinion.

Generating a draft time entry

At this point, Memory AI has a collection of memories, their predicted associated projects and is reasonably confident in its predictions. That said, we want to make sure you, the user, have the final say about how things are grouped before that time is logged.

As such, Memory AI groups memories according to their project predictions — providing the confidence is high enough — to create a draft time entry. You can either leave your draft as it is or amend it before hitting “Approve” to make it timesheet-official.

🍰 It’s that simple! In case you have any specific questions related to any of the above, don’t hesitate to nerd out with us — just send your message to [email protected].

⚠️ ML Answers (not that you needed them)

Problem 1

y = x + 1

Problem 2

y = x + x / 2

Problem 3

y = 8 ∗ max(0, 2x + 5) + 2 ∗ max(0, 3x + 4) + 6

Did this answer your question?