SpriteDX - Anti-Corruption Model v1.1.2

With v1.1.1, the model is now able to use the input alpha when non-trivial alpha channel is provided.

The resulting sprite shows quite a bit of high frequency noise.

Input Image:

Current Result:

In v1.1.2, I want to solve the spottiness. I want smooth parts to be smooth with clear edges.

Trial #1

We added gradient loss.

# Gradient preservation loss
def compute_gradients(img):
    grad_x = img[:, :, :, 1:] - img[:, :, :, :-1]
    grad_y = img[:, :, 1:, :] - img[:, :, :-1, :]
    return grad_x, grad_y
out_grad_x, out_grad_y = compute_gradients(output[:, :3])
tgt_grad_x, tgt_grad_y = compute_gradients(target[:, :3])

grad_loss_x = (torch.abs(out_grad_x - tgt_grad_x) * mask_x).sum() / (mask_x.sum() + 1e-8)
grad_loss_y = (torch.abs(out_grad_y - tgt_grad_y) * mask_y).sum() / (mask_y.sum() + 1e-8)
grad_loss = grad_loss_x + grad_loss_y

Result: No visual difference was noticed.

Trial #2

We tried to tone down the noise level in our corruption pipeline. The Since our input data when inferencing is generated assets from AI generated arts. These often don’t contain those regular gaussian noises.

The reason why I added noise is because I was thinking noise would help displace the pixels a bit. However, it is creating more problem than it is worth it. So, I reduced it to 0.1x the original amount.

Result: Looks like it improves with a little bit of caveat that the highlights (whites) in the right hand side is little understated. But definitely an improvement.

More Samples:

Input:

Before:

After:

More:

— Sprited Dev 🐛