# SpriteDX - Loop Segment Detection

Yesterday, we resolved shot boundary detection and today I want to look into Loop Segment Detection.

Here is our current approach:

1. Given:
    
    1. Frames: f\_1 … f\_N
        
    2. min\_len: minimum number of frames.
        
    3. max\_len: maximum number of frames.
        
2. Grayscale each of them and down scale to 128×128.
    
3. Create composite frame `composite_i` such that:  
    R: Previous frame’s intensity<sup>.</sup>  
    G: Current frame’s intensity.  
    B: Next frame’s intensity.
    
4. For each (i, j) pair of frames (within min\_len, max\_len):
    
    1. Get mean squared error between the pixel values.
        
5. Pick the pair that has the lowest error, and return the segment `[i, j)`.
    

This worked reasonably well, but we faced some issues:

* **Temporal Cheats**: When min\_len is small, frames close to each other often has the least error just because they are close to each other temporally.
    
* **Metric Mismatch**: When picking pair A vs B, if A has less motion, there is higher likelihood that it would have lower MSE even though visual inspection reveals that higher MSE with more motion often looks smoother.
    
* **Repetitions**: When there is repetitions within a loop, there is no mechanism to find the smallest loop without repetition.
    

We tried addressing these by hand:

* To fight Temporal Cheats and Repetitions, we added length penalty.
    
* To fight Metric Mismatch: we adjusted error term using motion metric.
    

However, a lot of this was tuned by hand looking at the data and really can’t tell if they are working or not.

---

## Dataset

We are going to use the dataset we collected before. This contains character animations broken down into individual shots.

Dataset Size: **744**  
Each samples are labeled using the baseline approach discussed above.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1761084022412/c91b3aeb-be4e-4bf7-a139-03ce29fee8b2.png align="center")

Next, we need UI to review the labels and adjust them.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1761085602439/aae3361f-0482-4aac-8a83-3d94a9c768a0.png align="center")

---

Did some manual review of the loops. And from the looks of it. The loops generated seems rather high quality. The failures were mostly focused around bad input data.

* Input data does not have loop
    
* Longer loop sequence is discouraged (i.e. anything over 16 frames gets a penalty)
    
* Sometimes non-interesting loop segment is detected.
    

I think I will settle with current baseline for now.

---

## Samples

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1761089740103/855a0ce5-360e-42ae-8904-b38cc4d00b04.webp align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1761089770908/b47e9215-f480-4161-a877-b1406acb7bc9.webp align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1761089751405/97c12c2c-911d-43c5-bcc8-cd7ac0ae8fdf.webp align="center")

---

— Sprited Dev 🌱