Measuring the orientation of axi-symmetric particles, like cylinders or spheroids, can be quite challenging, but geometrical relations simplify the problem. When considering more complex shapes, like axi-symmetric and asymmetric rings, we can train Convolutional Neural Networks on synthetic data to perform this task.

During my post-Doc at the Aix-Marseille Université, I designed a pipeline to measure particle orientations in a linear shear cell with two perpendicular cameras. This approach is inspired by the LeNet-5, introducing two heads that process both top and side frames at the same times.

Two-headed LeNet pipeline
The pipeline is as follows: a given particle geometry, represented by a '.stl' file (a), is used as the basis for the generation of a synthetic data set in Blender (b). This data set is then used to train a Deep Learning model, with the objective of estimating the particle orientation given two perpendicular projections (c). A physical particle corresponding to the '.stl' file is also created through rapid prototyping (d) and employed in the experiments (e). Subsequently, the Watershed method is applied to the recorded data from the experiments prior to the Deep Learning model inference operation, which estimates the time-evolution of the three-dimensional particle orientation vector $\mathbf{n}$ in the given experiment.

Pipeline Overview

1. Input

The pipeline starts with the creating of a particle STL file with given charateristics, like particle shape and particle aspect ratio.

2. Two separated preparation branches

Starting from a particle STL file, two parallel branches are initiated:

  • Synthetic dataset generation: Using Blender, we render paired frames from two perpendicular camera views, similar to the experimental setup of our experiments. Each couple of images is annotated with the particle’s imposed orientation (the ground truth label). This kind of synthetic data is made available in this repository for selected particle shapes.
  • Experimental preparation: The same STL file is 3D printed to fabricate real particles for laboratory experiments.

Samples of coupled side-top frames for a ring with circular section and aspect ratio 0.45

Top view

Synthetic training dataset. Particle orientations are known.

Side view

Binarized recordings of one experiment.

3. Experiments

Particles are recorded simultaneously from two orthogonal views (top and side) while rotating in the viscous shear flow. These videos provide the raw experimental dataset, partially hosted in this repository. Minimal CV preprocessing is applied for particle tracking.

4. Training & Inference

  • A two-headed CNN, adapted from the LeNet-5 architecture, is trained on the synthetic dataset, learning to regress the particle’s 3D orientation from the two-view image pairs.
  • The trained model is then applied to the experimental recordings to infer the 3D particle orientation for each frame.

5. Output

The pipeline returns one time series of 3D orientations for each experiment, reconstrcuting the rotational dynamics of particles suspended in viscous shear flows.

LeRing: a Two-headed LeNet-5 implementation

You can find the Python implementation of our two-headed CNNs in this GitHub repository.

Data

I am uploading experimental recordings as well as synthetic datasets for multiple particle geometries in this Hugging Face repository. Researchers interested in this problem and inspired by our approach are invited to play with our data and contribute.