Metal 2 on the A11 GPU
Learn about the new GPU Family 4. Learn about the new Apple-designed GPU inside the A11 Bionic processor. Learn about the new features that Metal 2 brings for the A11-powered devices. Imageblocks, Tile Shading, Raster Order Groups, Imageblock Sample Coverage Control, Threadgroup Sharing. Also brifly look at Face Tracking on A11 using ARKit.
Working with Particles in Metal part 3
Trying a third approach to rendering particles, this time cranking up the number of particles rendered to millions! Using a random generator function to populate the window with particles. Learning about the new dispatchThreads(:) method in Metal 2. Updating positions and velocities on the GPU and drawing "thicker" particles using a basic neighboring formula.
Working with Particles in Metal part 2
Trying a different approach to rendering particles, this time moving away from distance fields to using vertex buffers. Cranking up the number of particles rendered to thousands! Using a Model I/O object as a blueprint for the object to be instanced in the vertex shader. Continuously updating the particles positions on the CPU and sending the updates to the GPU.
Working with Particles in Metal
Introduction to particle systems. Seeing how a particle is created. Keeping track of its position using a distance function. Using time to emulate gravity. Implementing basic collision detection.
Using ARKit with Metal part 2
Continuing with implementing the other stages in ARKit. Describing the features needed for Scene Understanding. Enabling plane detection in the ARKit app. Introducing the ARSessionDelegate methods for adding, updating and removing anchors. Creating the plane buffer, mesh and drawing it with a new pair of vertex and fragment functions.
Using ARKit with Metal
Introducing ARKit - the newest framework from Apple. Defining the three stages in ARKit - Tracking, Scene Understanding and Rendering. Describing the main ARKit classes used - ARSession, ARSessionConfiguration, ARFrame, ARAnchor and ARCamera. Looking into tracking states and interruptions. Implementing a full Metal renderer for ARKit.
Introducing Metal 2
Read about what's new in Metal, now called Metal 2. MPS has new features but most important - it now works on macOS too. See why the new Argument Buffers are the biggest addition to the framework this year. Read about how Raster Order Groups can help avoiding race conditions in shaders. Find out what the ProMotion and Direct to Display features are, and read about all the other new features introduced this year.
Working with memory in Metal part 2
Looking again at the memory models in Metal. Analyzing data copy vs no-copy methods. Understanding stack vs heap allocations. Page aligning the data using posix_memalign(). Looking again at the resource storage modes - Shared, Private, Memoryless and Managed. Considerations about resource coherency and synchronization between CPU and GPU.
Working with memory in Metal
Learning about the memory model in Metal. Defining Metal resources as Buffers and Textures. Presenting ways to create a Buffer. Defining the Resource Storage Modes. Presenting the address space qualifiers for function variables and arguments. Looking at UnsafePointer and Pointee as another way to aceess memory locations.
Ambient Occlusion in Metal
Learning about Ambient Occlusion in Metal. Using more distance functions - rectangular shapes. Introducing cone tracing as a derivative of ray tracing using "thicker" rays. Using spheres with different radii to approximate a cone. Marching along the cone instead of a ray, to determine how much light is occluded. Replacing the circling light source with a moving camera so we can examine the entire scene.
Shadows in Metal part 2
Learning about Soft Shadows in Metal. Returning to Raymarching to build a complex 3D scene using distance operation functions. Learning how to calculate normals and the visualizing them. Learning how to use normals to calculate diffuse and specular lighting. Determining hard shadows as well as soft shadows for objects in a 3D scene.
Shadows in Metal part 1
Learning about Shadows in Metal. Building custom functions and operators such as modulus. Creating a light source and animating it around the scene. Creating a shadows function based on the distance and direction to the light. Marching along the light ray and improving the number of steps needed for faster shadows.
Raymarching in Metal
Learning about Raymarching in Metal. Continue on improving previously used Signed Distance Fields functions. Seeing the similarities and differences from Ray Tracing. Using the ray as camera and marching along with it. Instancing an object to create more complex scenes.
Beginning Metal - a new course
Learning about the Metal API in the form of a video course that has no less than 15 chapters with well structured material, code samples and challenges to solve. Building your first game completely written in Swift and Metal.
Using MetalKit part 2*3^2
Prototyping kernels using math functions and the Swift Playgrounds app for iPad. Using sqrt to shape transitions, then cos to create variations of ups and downs in shapes, then exp that allows us to create curves, then smoothstep for thresholding, then abs for symmetry and mix for blending.
Using MetalKit part 17
Learning how to draw wireframes by setting the triangle fill mode to lines. Disabling the back-face culling for 3D rendering so we can see all the wires, including those in the back.
The Model I/O framework
Introducing Model I/O - a framework that helps us create more realistic and interactive graphics. Learning how we can use it to import/export 3D assets, to describe lighting, materials and environments, to bake lights, to subdivide and voxelize meshes, and for physical based rendering.
Ray tracing in a Swift playground part 6
Porting the Raytracing code to the Swift Playgrounds app for iPad. Tweaking the number of samples for getting better quality images vs faster rendering. Adding a viewer for the raytraced content.
Metal Performance Shaders for the iPad playground
Learning how to easily import a playground from a Mac to an iPad using the iCloud Drive. Seeing how a MPS object does the command encoding directly, without having to configure any states, descriptors or functions.
Using MetalKit part 16
The Metal Best Practices Guide offers advices for Resource Management (persistent objects, resource options, triple buffering, buffer bindings), Display Management (drawables, native screen scale, frame rate), Command Generation (load and store actions, render command encoders, command buffers, indirect buffers) and Compilation (functions and libraries, pipelines).
Using MetalKit part 15
Learning how textures can provide object surfaces a greater level of detail. Using the texture loader function to import an image to be used as texture. Fixing the flipped upside-down texture. Introducing samplers that offer advanced control over texture coordinates through addressing modes and filtering methods.
What's new in graphics and games at WWDC 2016
Metal gets important updates such as Tessellation, Function Specialization, Resource Heaps, Memoryless Render Targets and the Metal System Trace. Also see what's new in Model I/O, SceneKit, SpriteKit, GameplayKit, ReplayKit, Accelerate and CoreImage.
Using MetalKit part 14
Understanding basic value noise and how randomness plays an essential part in generating it. Creating a custom randomness function. Bilinearly interpolating a grid using a Cubic Hermite Spline and a Fractional Brownian Motion function. Add the timer uniform again to animate the noise content.
Using MetalKit part 13
Moving on to 3D objects in the kernel function. Using the sphere equation to get distances to points in space. Calculating normals calculating and then the color at each normalized coordinate. Learning about diffuse (Lambertian) basic light. Using uniforms again to animate the light source position around the sphere.
Using MetalKit part 12
Continue working on the fractal from the previous part. Using uniforms to send the GPU updated information about a timer and about the mouse click coordinates. Blending the color with the current coordinates to achieve new effects.
Using MetalKit part 11
Using MSL math functions such as sin, cos, pow, abs, fmod, clamp, mix, step and smoothstep. Understanding how smoothstep helps us fix graphics challenges and issues such as aliasing. Seeing how abs and fmod help us draw repetitive patterns such as lines of various slopes and thickness. Using trigonometric functions such sin, cos, fract, dot and pow to generate fractals.
Using MetalKit part 10
Introducing the compute pipeline and functions (kernels). Using MTLSize to determine the size of a threadgroup and the number of groups in a grid. Dispatching the threadgroups to do work in parallel. Using the thread_position_in_grid index to identify each thread. Working with the texture coordinates and dimensions. Introducing distance functions.
Using MetalKit part 9
Moving to 3D rendering. Introducing index buffers that are useful for vertex reusing. Presenting the front-facing winding orders and the culling modes. Introducing the concept of depth. Building a model-view-projection matrix. Describing the viewing frustum. Using rotation updating each frame to create an animated spinning effect.
Ray tracing in a Swift playground part 5
Creating a random scene with a few dozen randomly textured and positioned spheres. Using Grand Central Dispatch threading to speed up rendering times or create images of higher quality by increasing the number of samples.
Ray tracing in a Swift playground part 4
Creating a dielectric material. Using the Schlick polynomial approximation for higher reflectivity. Applying Snell's law to compute refraction. Tweaking the normals to get a hollow glass surface. Fixing the camera to get better views from different angles and distances.
Ray tracing in a Swift playground part 3
Creating an abstract material protocol that has a scatter function. Creating a lambertian and a metallic class that conform to the material protocol. Recursively computing color definition until a given depth is reached.
Ray tracing in a Swift playground part 2
Switching from our custom built data structures to the more performannt SIMD ones. Creating a timing function to monitor our rendering performance. Noticing that we can easily avoid copying large arrays. Creating two more structs (hit_record and camera). Creating a hitable protocol (with a hit function) and a hitable list. Implementing anti-aliasing and diffuse lighting.
Ray tracing in a Swift playground
Introducing Raytracing. Creating the basic building blocks - a pixel struct and a ray struct. Populating an array of pixels with color and rendering into an image object. Overriding a few basic operators. Creating a hit function and using it in another function that is meant to determine color based on found intersections. Using the color function for each pixel in our view.
Using MetalKit part 8
Introducing Metal prototyping in the Swift playgrounds. Making sure the Swift source code is created in the Sources directory while the Metal files (as well as any assets we might have) are created in the Resources directory. Creating the library from a path since a default one cannot be created in a playground.
Using MetalKit part 7
Transient vs non-transient Metal objects. Making sure all the Metal objects are reused in multiple draw calls except for the command buffers and encoders. Creating a utilities class for separating responsibilities the view controller should not have.
Using MetalKit part 6
Looking at the differences between using the Metal and MetalKit frameworks. MetalKit provides easy texture loading, a Metal-aware view and interfacing with Model I/O.
Using MetalKit part 5
Introducing 2D/3D transforms - scaling, translation and rotation matrices. Using a Model matrix to combine all transforms and sending it to the GPU as a Uniform using a data buffer. Introducing argument tables for sending multiple data buffers to the GPU. Using the model matrix in the vertex shader for transformations.
Using MetalKit part 4
Looking at the graphics pipeline in more detail. Building a Metal application in two stages - Initialization (get device, create command queue, create buffers/textures, create render pipeline, create view) and Drawing (get command buffer, start render pass, draw, commit command buffer).
Using MetalKit part 3
Using structs of vectors for vertices. Vector swizzling in MSL. Synchronizing struct data between the CPU and the GPU. Vertex and Fragment function syntax - function qualifiers, return types, address space qualifier, attributes. Interpolating color using the information stored in the vertex structs.
Using MetalKit part 2
Introducing 2D geometry and using more Metal objects - data buffer, library, function, render pipeline descriptor and state. Introducing the Metal Shading Language. Drawing a triangle on the screen.
Using MetalKit part 1
Introducing MetalKit and its specialized Metal view - the MTKView. Using the Metal objects - command queue, render pass descriptor, command buffer, command encoder and drawable - to clear the screen to a custom color.
Introducing the Metal framework
What is Metal? What are its advantages over other frameworks? Which GPUs are supported by Metal? How to initialize a Metal device.