As many of you might have seen at WWDC 2016, the new Playground app for the iPad was a really big hit! As an already playgrounds lover, for me it was even more than that. Now we are able to easily write Swift code on the iPad and run it with just a button tap. Today we are going to have fun with Metal Performance Shaders (MPS) since I have not discussed about them before, and also because it is so easy to use them on mobile devices. This reminds me to warn you, the MPS framework only works on iOS and tvOS devices. We will look into a handy way of writing the playground on a macOS device and then sharing it with your mobile device through the iCloud Drive. Also, the iPad playgrounds are working only on iOS 10 or newer.
To start, let’s create a new iOS playground in Xcode. You can add any image you like under the Resources folder. I already added one named nature.jpg. Next, write a few lines of code in the main playground page, like in this screenshot (the code is available on Github):
Let’s go over the code. We have been writing most of this code over and over in the previous blog posts. The first new addition you will notice immediately is also generating an error, and that is because macOS does not “know” about the MPS framework. Once we load this playground on an iPad, the error will go away:
import MetalPerformanceShaders
Next, we use MTKTextureLoader to create a new texture from the image we added above. Now comes the really interesting part! Once we created our MTLCommandBuffer object, we are not going to create a MTLCommandEncoder object too from this command buffer as we were used to do. Rather, we create a new MPSImageGaussianBlur object as in the code below:
let shader = MPSImageGaussianBlur(device: view.device!, sigma: 5)
shader.encode(commandBuffer: commandBuffer, sourceTexture: texIn, destinationTexture: texOut)
What’s great about the MPS objects is that they let you apply a compute shader (kernel function) to an input texture without us even having to configure any states, descriptors, pipelines or even write a kernel function, ever! The MPS object takes care of everything for us. Of course, by taking this approach we are somewhat limited to only picking a preset shader and possibly changing a parameter such as sigma, for this particular shader.
So far so good! We are now ready to send our playground to the iPad through the iCloud Drive. Open a Finder window, click on iCloud Drive and copy your playground into this folder:
We are finally getting to use the iPad! Open the new Playgrounds app that you downloaded from the App Store and go to My Playgrounds. In the top left corner of the screen, tap on the + button. As you can see you can also create a new playground on the iPad and then easily export it to your macOS devices or share it using various other tools. For now, however, we are going to tap on iCloud Drive as seen below:
When the iCloud Drive window pops up, notice that we have access to the iPad’s private folder for playgrounds as well, however, we now want to import the one we were working on, MPS.playground, so tap on it:
As soon as the iPad loaded the playground, we’re in business! You can see the main playground page now on your iPad:
All you have to do now is tap on Run My Code and see our image with a nice blur filter applied to it:
“Ok,” you might say, “but how do we see the entire image?” The answer is in the animated GIF below. It’s as easy as long pressing in the middle of the screen until a screen separating line shows. With the finger still on the screen, drag the line to either the left side or the right side, depending on what do you need to see, your code or your output image (you might want to reload the page as the animated GIF may only run once before it stops):
Now you can see that blurred image entirely! And when you’re done admiring the image, tap that handy button on the left side (and midway vertically) of the screen to go back to split screen. Before we wrap it up, one more piece of awesomeness. Replace the line below:
let shader = MPSImageGaussianBlur(device: view.device!, sigma: 5)
with this line:
let shader = MPSImageSobel(device: device)
Check out the new output image:
Imagine the possibilities you have here by only changing one line of code! There are a couple dozen different shaders that you can try. Check out the Metal Performance Shaders API for more details. If you are interested in image processing, you might want to also check Simon Gladman’s Core Image for Swift book. If you want to learn more about the Metal backend of MPS, also check out Warren Moore’s Metal by Example book. The source code for the playground in this article is posted on Github as usual.
Until next time!
Everything about the Metal API 