Development Blog

I’ve already mentioned this briefly on Twitter, and there was some interest in the topic so I’ll give a brief overview of the tech stack that I’m employing for Hyperdeck.

I’ll keep this brief as I will supplant it in the coming weeks with more detailed posts about the individual pieces of the stack.

It was of uttermost importance to me, that the typing & evaluation loop in Hyperdeck is always very very fast. You should see the changes in your Markdown rendered in the preview as fast as possible. Also, it should still be fast on older iPads. So, the focus was always on using a native technology stack - without Javascript or Webviews. This is important as it drives a couple of fundamental decisions.

The Hyperdeck UI can really be seperated into three main sections (with several supporting sections which I will conveniently ignore now)

1. The Editor
2. The Slide Inspectors
3. The Slide Preview

The Editor

I wanted to have a native editing experience, and this is where UIKit shines. Since iOS 7, Apple has been shipping the full text editing stack that has been powering macOS for years. With UITextView, there's a default implementation that can be adapted via subclassing and NSTextStorage, NSTextContainer and NSLayoutManager.

I wanted to do some very unusual things here, which took me some time to figure out, mostly due to the lack of documentation or sample code. I also had some particular problems relating to the fact that the truth for the data was Markdown in a Rust model (see below) using a different text encoding and different index encoding.

Initially I was afraid of performance problems when applying a lot of attributes, but after rewriting parts of the code in Objective-C - due to this Swift issue, the performance has been consistently great.

One my main gripes right now is that the macOS version of Hyperdeck runs on Catalyst and the Catalyst version of UITextView has multiple issues. Thankfully, Peter Steinberger has done what Apple apparently couldn’t, and released code that fixes a particular nasty crash in Catalyst UITextView apps.. So now, at least the Catalyst version of Hyperdeck doesn’t crash anymore when writing text.

The other very problematic issue that remains is that on Catalyst, you can’t disable Smart Quotes and Smart Dashes, so whenever a user types --- (three dashes), it will be replaced with a —. This makes it tricky to write slide separators or Markdown tables.

The Slide Inspectors

The Inspectors are the small popovers that allow modifying Markdown elements. Changing the size of a headline or the alignment of a slide, or the theme of a code block.

These inspectors are completely written in SwiftUI. My initial reason for doing this was that I wanted to use SwiftUI Forms to build this. However after discovering that they're really not any good I went on and implemented my own Inspector Form library on top of SwiftUI (and some UIKit where SwiftUI falls short, but that's a story for another blog post). Now, I can define a Inspector like so:

struct Inspector: View {
    ...
  public var body: some View {
    // Create a new inspector view
    InspectorView {

      // This cell should be expandable
      ExpandableCell(
        icon: Icon.innerAnimation,
        title: "Duration",
        subtitle: self.baseState.animationDurationDescription) {
          // The expanded contents
          Title("Duration")
          InspectorDurationView(duration: self.$baseState.animateDuration)
      }

      // This cell should push into a detail view
      InspectorDetailViewCell(
        // The Dteail View
        InspectorListSelectionView(
          "Select Animation",
          entries: self.baseState.animationCases(),
          selected: self.baseState.animateType) { animation in
                Text(animation.title)
      }) {
        // The cell entry that opens the detail view uppon tapping
        Text(self.baseState.animationTitle)
      }
      // If we have an animation, display a reset button
      if self.baseState.hasInnerAnimation {
        Button("Reset Animation") { self.baseState.resetAnimation() }
      }
    }
    // This cell has a fixed height and custom content
    CustomCell {
      FontSizeSlider("Font Size", fontSize: self.$baseState.fontSize)
    }
  }
}

(simplified example, the actual code has more warts)

What I really like is how this allows me to define the UI in the same way as it appears on screen. With UICollectionView, a similar UI is oftentimes split up into Delegate, DataSource, Cells, Custom Layout, View Controller(s), and decoration views. This means that when I want to change something I have to edit ~5 types in ~4 different files. This, to me, is one of the main benefits of SwiftUI: Much easier iteration.

Looking back, there were several problems with this approach, almost all related to missing or buggy features in SwiftUI. I have high hopes for SwiftUI 2.0 (or whatever it will be called) to remedy these issues, allowing me to greatly simplify my implementation. If not, I have material for multiple blog posts.

My favorite issue shall be listed though: I have a situation where taps don’t work on SwiftUI Buttons in a SwiftUI ScrollView. Initially I was puzzled because sometimes they worked, sometimes they didn’t. After some messing I figured it out. The buttons only work if the ScrollView has at least been scrolled one pixel. I will not go into the terrible detail of how that is currently fixed, but it the solution is based on the fact that the ScrollView is actually a UIScrollView so there is a way of locating it in the hierarchy.

The Slides

The inspectors are not the only part written in SwiftUI. The major part of Hyperdeck, the slides, are also a mostly SwiftUI implementation. I say mostly because, as with everything in SwiftUI, it is complicated. That's a story for another blog post though.

The way this works is that each slide is one UICollectionViewCell hosting a UIHostingController. Each controller than contains a SlideView View instance. When the Markdown for a slide is modified, a new Slide struct is generated and applied as a model to the SlideView.

This way, I did not have to build up a complex diffing solution to figure out which slides changed, because:

1. I'm using Diffable Data Sources. So when new slides are generated, the data source makes sure to only reload the parts of the collection view that changed
2. SwiftUI has built-in diffing. When only a headline on a slide changes, SwiftUI will not re-render the rest of the slide, only the headline

It took me some time to figure out how to structure the slides and the collection view cells so that this works consistently. Also a thing I'll hopefully write about in a future post.

Another issue I ran into was figuring out how to best scale the slides as the user resizes the preview. Initially I tried using the scale modifier, but that had too many limitations. The current setup is calculating a ratio which is applied to all elements.

As is the standard with SwiftUI, this also had a lot of problems, particularly when it came down to text. I have another blog post planned for this, but the gist is that SwiftUI doesn't support attributed / rich text. The best solution is to host UILabel in UIViewRepresentable, but that has terrible performance. Beta 1 and 2 of Hyperdeck had terrible scrolling performance due to this. I've since found a ok’ish solution (and will write about that soon), but I hope that WWDC 2020 will introduce a much better and more performant way of solving this.

Parsing Markdown

When I started with Hyperdeck, in August 2019, there was no native Markdown parser for Swift. I could have wrapped CMark, but it is written in C. Since I knew that I wanted to extend the Markdown syntax with additional properties, I was afraid this would only introduce difficult-to-reason-about bugs. Here's an example of the additional Markdown syntax

:: size: 150, master: title2
# Hello World

--- :: align: right

I also did not want to use a Javascript Markdown parser because of the overhead of calling Javascript from Swift.

Another issue was that I was planning on having only a single parsing pass. Basically write Markdown, parse the Markdown and then use the parsed information to:

1. Format the UITextView with attributed strings
2. Build a SwiftUI View for the Slide Preview
3. Support Commonmark, the official Markdown spec

Some Markdown parsers just parse Markdown to HTML. This wasn’t helpful to me because I needed to attribute text and build a SwiftUI hierarchy. Some Markdown parsers just format text (oftentimes via Regular Expressions, which is quite slow). Finally, some Markdown parsers only support a very limited subset of Markdown (which is actually a rather complex format… check the headline examples here.) I really needed something that would give me information about the Markdown, something like ”There’s a headline byte index 142 to byte index 255”:

let markdown = "# Hello **world**"
struct Headline {
  let elements = [
      Headline(
          elements: [
             Text(range: 3..<8), 
             Bold(range: 9..<14)
          ]
      )
  ]
}

Thankfully, there was a Rust library for this: pulldown-cmark.

For each Markdown element, it also returns the offset, making it easy for me to know the actual locations in the Markdown. It is also very fast, even using SIMD operations.

Finally, since I was already fluent in Rust, it was also easy for me to extend it to support my additional syntax (and possible future additions). As you can imagine, adopting a mixed Rust/Swift project wasn’t without pain, but that’s a story for a different post.

Nowadays, there’s a native Swift Markdown Parser of course. However, it still only renders HTML, so it would also have been additional work for me to expand it to return the offsets that I require. Using Rust had another advantage: There’s also a beautiful Rust library for syntax highlighting (using Sublime Text syntax definitions). This means that my single parsing pass also parses any source code snippets on the slides and returns syntax definitions that are used to do the syntax highlighting on the slides and in the editor. This leaves Rust with the following responsibilities:

• Parse Markdown
• Highlight Code
• Parse & understand the element options (::)
• Modify Markdown (Delete slides, add size options)
• Table operations (format tables, insert rows, delete cols, etc)

Regarding speed, the current library (named Parseval) parses and highlights 400 slides on my Mac mini in ~500ms. Subsequent runs (basically editing a slide) take < 3ms as the first step builds up a cache.

Looking back on the tech decisions

I’m still very happy that I chose Rust for the Markdown parsing. Not only is it fast, it also looks a lot like Swift, I have to care less about memory issues, and there’s a great ecosystem of fantastic packages that I could hook into to solve problems without having to re-invent the wheel.

SwiftUI was an easy sell as it sounded like the perfect solution for the kind of slides I had in mind. In general that’s still the case, however two things stick out which I hope will be solved soon but are currently tricky:

1. Missing support for attributed strings which cause performance penalties
2. Crashes. I haven’t talked about this yet, but during the beta test I’ve seen an uncanny amount of crashes in SwiftUI code come in. Usually, those crashes have no Hyperdeck in their stack trace (except for AppDelegate) so it is tricky to figure out what I’m doing wrong. The most crashes seem to happen for users with the new Magic Keyboard that interact using the touchpad cursor with SwiftUI views. My hope is that the new SwiftUI will perform much better here.

As mentioned throughout this post, I have many more blog posts planned, but this is a good overview explaining the general tech stack. Also, the whole blog post was written in Hyperdeck.