Pixel Art Characters

Trained a custom LoRA to transform real photos into game-ready pixel art characters while preserving facial identity across multiple styles.

---

Problem

Most AI-generated pixel art characters struggle with face consistency and the output often loses recognizable details of the subject. I wanted to solve this by training a workflow that could:

• Convert a normal photo into a pixel art game character.
• Maintain identity fidelity across multiple outputs.
• Adapt the character to different gaming backgrounds and styles.

Approach

I designed a two-stage pipeline combining custom LoRA training with pixel art style transfer inside ComfyUI.

1. Data Collection

   • Gathered images from Instagram reels and static posts.
   • Built a training dataset of diverse facial shots.

2. LoRA Training

   • Trained a Flux LoRA on the dataset using ai-toolkit.
   • Used trigger word “MSD” to enforce face consistency.
   • Tested checkpoints at 2500, 2750, and 3000 steps, finalizing 3000 steps for optimal results.

3. Environment & Tools

   • Deployed a GPU instance on JarvisLabs.
   • Imported required ComfyUI nodes and dependencies.
   • Combined the face LoRA with a pre-trained Pixel Art Flux LoRA for consistent style transfer.

4. Generation & Iteration

   • Ran multiple iterations, fine-tuning parameters for stable results.
   • Ensured facial identity remained intact while achieving authentic pixel-art aesthetics.

Tools & Stack

• Training: ai-toolkit, Flux LoRA (3000 steps)
• Environment: JarvisLabs GPU instance
• Workflow: ComfyUI with custom + pre-trained nodes
• Style Transfer: Pixel Art Flux LoRA

Outcome

• Generated 6 high-quality pixel art game characters.
• Maintained consistent facial identity across outputs.
• Built a reproducible workflow to adapt any photo into a game-ready pixel art character.