Federated Learning and data privacy on your keyboard.

Story by Lucy Bellwood and Scott McCloud. Art by Lucy Bellwood.

Introduction to Federated Learning. The what & why

Why Federated Learning.

What if the model is deployed on-device?

Federated Learning allows us to keep enhancing models in the current user’s device, keeping the user’s raw data and learning on their device.

How Federated Learning works

  • A server with its own data and initial trained model.
  • The user’s device with its own data stored inside.

The idea behind Federated Learning is to train a centralized model based on decentralized data

  • The server will prepare the initial model and push it on the device to download it.
  • Then, the device starts training starting from that model and based on the data that it has in its own little data store.
  • Finally, the device uploads this new model to the server.
Image taken from Google’s YouTube video on Federated Learning

AI and Federated Learning on mobile keyboards.

Keyboards nowadays have functionalities that use machine learning as their engine.

Image taken from Google’s YouTube video on Federated learning: Basics and application to the mobile keyboard (ML Tech Talks)

Process overview

  • The first step is that we come up with an initial model on the server-side selecting clients where the training rounds will happen.
  • Once the training part in the client-side is done, the client returns the updated set of weights or the updated model to the server.
  • Finally, the server combines all the models into a new model.

The initial model

  • Consumes less privacy budget.
  • It takes less time for reaching the same accuracy. In the example that Françoise Beaufays provides, for reaching the 15% of accuracy:
  • Without an initial model: It takes 350 rounds of training.
  • Using an initial model: It takes 80 rounds.

Optimization in the client side

Federated learning needs a lot of previous planning before implementation.

  • The hyper-parameters need to be optimized for a fast convergence (batch size, epochs, learning rate).
  • The data (amount & quality) that each device will have will be really different compared with other devices (different usage, different data).

Requirements for the keyboard training (GBoard case)

Image taken from Google’s YouTube video on Federated learning: Basics and application to the mobile keyboard (ML Tech Talks)

Real world applications

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Juan

Juan

Lead Data Scientist 📊 at Bravo Studio. Start-up growth advisor 🚀 He/Him — 🏳️‍⚧️🏳️‍🌈 ally. Opinions are my own