Run Stable Diffusion using Core ML on iOS & iPadOS within your Expo & React Native App
npm install expo-stable-diffusion!GitHub last commit
!npm
!GitHub issues
!GitHub stars
> ❗️ expo-stable-diffusion currently only works on iOS due to the platform's ability to run Stable Diffusion models on Apple Neural Engine!
> ❗️ This package is not included in the Expo Go. You will have to use a Development Build or build it locally using Xcode!
Start by installing the expo-stable-diffusion module into your Expo managed project:
``bash`
npx expo install expo-stable-diffusion
In order for the project to build successfully, you have to set the iOS Deployment Target to 16.2. You can achieve this by installing the expo-build-properties plugin:
`bash`
npx expo install expo-build-properties
Configure the plugin by adding the following to your app.json:
`json`
{
"expo": {
"plugins": [
[
"expo-build-properties",
{
"ios": {
"deploymentTarget": "16.2"
}
}
]
]
}
}
To prevent memory issues, add the Increased Memory Limit capability to your iOS project. Add the following to your app.json:
`json`
{
"expo": {
"ios": {
"entitlements": {
"com.apple.developer.kernel.increased-memory-limit": true
}
}
}
}
`bash`
npx expo prebuild --clean --platform ios
npx expo run:ios
After installation and configuration, you can start generating images using expo-stable-diffusion. Here's a basic example:
`typescript
import * as FileSystem from "expo-file-system";
import * as ExpoStableDiffusion from "expo-stable-diffusion";
const MODEL_PATH = FileSystem.documentDirectory + "Model/stable-diffusion-2-1";
const SAVE_PATH = FileSystem.documentDirectory + "image.jpeg";
await ExpoStableDiffusion.loadModel(MODEL_PATH);
await ExpoStableDiffusion.generateImage({
prompt: "a cat coding at night",
stepCount: 25,
savePath: SAVE_PATH,
});
const subscription = ExpoStableDiffusion.addStepListener(({ step }) => {
console.log(Current Step: ${step});
});
return () => subscription.remove();
`
> 💡 If you are saving the image in a custom directory, make sure that the directory actually exists or you can create it using FileSystem.makeDirectoryAsync(fileUri, options).
To use the expo-stable-diffusion` module, you need a converted Core ML Stable Diffusion model. You can convert your own model using Apple's official guide or download pre-converted models from Apple's Hugging Face repository or my Hugging Face repository.
> ❗️ The model load time and image generation duration take some time, especially on devices with lower RAM than 6GB! Find more information in Q6 in the FAQ section of the ml-stable-diffusion repo.