A few weeks ago, one of my coworkers came across a bug in the Swift compiler. The bug is filed as SR-13815 and deals with a strange interaciton between optionals and implicit member expressions.

Another coworker asked how exactly I went about tracking down the root cause of this bug, so I decided to write up a summary of my process!

The bug

The following is a minimal reproducer for the bug, stripped of any dependence on external libraries like SwiftUI (which is where the bug was originally noticed):

extension Optional {
    func foo(_: Int) {}
}
    
struct S {}
    
extension S {
    static let foo: S? = S()
}
    
func takesOptS(_: S?) {}
    
takesOptS(.foo) // Error: Value of optional type 'S?' must be unwrapped to a value of type 'S'

The error message here is clearly nonsensical. After all, takesOptS accepts an argument of type S?, so unwrapping should be completely unnecessary. The presence of the foo(_:) function in our Optional extension hints at the cause, but imagine this were defined in a different file somewhere. The error message doesn’t mention foo(_:) at all, so at first glance it’s very hard to understand what’s going on.

Digging in

So, how should we approach this issue? Thankfully, the Swift compiler has some built-in debugging functionality to understand errors like the one above. Generally, anything that deals with type mismatches comes out of some failure during type-checking, which means that we can use the -debug-constraints flag to figure out what’s going on.

Using the release version of the compiler, we can invoke just the type checker with debug output enabled with the following command:

$ swiftc -Xfrontend -typecheck -Xfrontend -debug-constraints /path/to/file.swift

The output from this command is pretty dense, but it summarizes the whole process of type-checking the specified file. You can identify the particular expression via the headings:

---Constraint solving at [/path/to/file.swift:<line>:<col> - line:<line>:<col>]---

In our case, we’re looking for an expression on line 13, and indeed:

---Constraint solving at [/path/to/file.swift:13:1 - line:13:12]---

The actual output for this expressions contains every type that the compiler attempts to assign to various parts of this expression. That means that if we want to figure out where it’s going wrong, we should have an idea of the expected type of each part of the expression in question. Let’s look at it again:

takesOptS(.foo)

Luckily for us, this expression is pretty simple. There’s really just four parts that we want to think about. Let’s proceed in a ‘top-down’ manner:

  1. The type of the entire expression, that is, the result of calling takesOptS. This should clearly be ().
  2. The type of the function takesOptS. There’s no fancy overloading going on, so this is also straightforward. It should be (S?) -> ().
  3. The type of the implicit member expression .foo. We want this to refer to S.foo if everything is working properly, so it should have type S?.
  4. The type of the implicit base of .foo. Again, since we want .foo to refer to S.foo, the type of the base should be S.

For each of these sub-expressions, the compiler creates a “type variable” for which it will try different type substitutions until it finds an assignment for all type variables that works. The next lines of the output shows the type variable assignments (edited to remove noise):

(call_expr type='()' arg_labels=_:
  (declref_expr type='(S?) -> ()' decl=takesOptS)
  (paren_expr type='($T3)'
    (unresolved_member_expr type='$T3' name='foo')))

(Note: the compiler calls implicit member expressions “unresolved member expressions”)

We can see right off the bat that the result of the call and the type of the function reference takesOptS have already been resolved to () and (S?) -> (), just as we expected. Great!

We can also see that the implicit member expression .foo has been assigned type $T3. This is how the debug output represents a type variable with ID 3. The ID has no effect except providing a unique name for the type variable.

Because it isn’t written in the source, the type of the implicit base doesn’t get printed here. But if we look just below to the list of all type variables, we can see the following entry:

$T1 [noescape allowed] potentially_incomplete involves_type_vars #defaultable_bindings=1 bindings={<<unresolvedtype>>} [UnresolvedMember -> member reference base]

We don’t need to be concerned with most of this output—the only important part is UnresolvedMember -> member reference base, meaning that this type variable represents the base of the implicit member expression.

Okay great! So now, with our knowledge of the type variables involved and the expected values for each, we can analyze the output further. It seems that the error message is having trouble with the type of the .foo expression, so let’s look for output involving $T3:

(attempting type variable $T3 := S?
  ($T2 involves_type_vars bindings={(subtypes of) S?})
  (attempting disjunction choice $T2 bound to decl bug.(file).Optional extension.foo
    (overload set choice binding $T2 := (S?) -> (Int) -> ())
    (increasing score due to value to optional)
    (failed constraint $T2 conv $T3)
  )
  (skipping disjunction choice $T2 bound to decl bug.(file).Optional extension.foo [fix: allow access to instance member on type or a type member on instance])
)
(attempting type variable $T3 := S
  (increasing score due to value to optional)
  (overload set choice binding $T2 := S?)
  (failed constraint $T2 conv $T1;)
)

(Note: $T2 here also corresponds to the type of the expression .foo—because of the way the constraints get generated, $T2 is basically “the type of the member”, and $T3 is “the type of the overall expression. This allows certain conversions to take place that improve the ergonomics of implicit member expressions, but is mostly unimportant here.)

Interestingly, when $T3 is substituted out for the “correct” type S?, the compiler only attempts to match foo with Optional.foo(_:), the method defined in our extension. It’s only when we attempt $T3 := S that the compiler finds the expected member, S.foo.

So now, we know that this issue has something to do with how the compiler is looking up the .foo member. To debug further, we need to look at the compiler’s code itself.

The ConstraintSystem

All of the type checking happens within a component of the compiler known as the ConstraintSystem. This class is responsible for creating all the type variables for the expression, generating “constraints” between the type variables based on the expected relationships, and then “solving” the system by finding an assignment of types to type variables that satisfy all constraints. For instance, an expression like:

S.foo

Would generate a constraint that says “the type referred to by the name S must have a member named foo with type _” (where the desired type is determined by context).

This type of constraint is called a “value member” constraint, and is represented by the constant ConstraintKind::ValueMember. There is also a corresponding “unresolved value member” constraint and ConstraintKind::UnresolvedValueMember constant.

With some preexisting knowledge of the Swift compiler (or some debugging work), we can track down the fact that the actual name lookup for both of these constraint kinds happen in a method called ConstraintSystem::simplifyMemberConstraint. When constraints are “simplified,” they are converted into simpler sets of constraints (or eliminated entirely) based on other context that has been filled in previously. For instance, with an implicit member expression like .foo, we can’t simplify the constraint until we know what type is supposed to be filled in the implicit base.

Once that context has been filled in, though, we can proceed with simplification. Looking through the source of simplifyMemberConstraint, we can find the following line which looks interesting:

  MemberLookupResult result =
      performMemberLookup(kind, member, baseTy, functionRefKind, locator,
                          /*includeInaccessibleMembers*/ shouldAttemptFixes());

Indeed, this is the point at which the compiler actually looks into the base type to find a member named foo. Opening the source for this method, we can see that it’s very (very) long, but if we know what we’re looking for we don’t have to analyze every single line.

Aside

There’s something a bit strange going on with the implicit member expression. Conceptually, an implicit member expression like .foo does the following:

  1. Inherits the expected type from the surrounding context.
  2. Substitutes that type at the base of the member reference.
  3. Looks for the (static) named member in the base with the correct type (same as the base).

That is, if we write .member in a location where the compiler expects a type T, that should basically be the same as writing T.member.

However, that doesn’t really account for what’s going on with an implicit member expression involving optionals. For instance, if we write:

let image: UIImage? = .init(named: "myImage")

Then both the contextual type and the result type of the .init implicit member expression have type UIImage?, but we can’t just replace the implicit member expression with UIImage?.init(named: "myImage"), because Optional<UIImage> has no init(named:) initializer.

Indeed, implicit member expressions have a little extra magic. For an expression .member with contextual type T, if T is an optional type U?, then the compiler will look for member in U as well as in T. However, based on the -debug-constraints output, it looks like that’s not happening. The compiler never attempts the foo member that it should find in S—it only attempts to use Optional.foo(_:).

Back to work

Armed with this knowledge of the interaction between optional types and implicit member expressions, we have a plausible guess for what’s happening: the compiler is looking for the foo member in Optional<S>, finding Optional.foo(_:), and not continuing look into S to find S.foo.

Let’s look a little closer at the ConstraintSystem::performMemberLookup method. The first interesting line is:

      // Look for members within the base.
      LookupResult &lookup = lookupMember(instanceTy, memberName);

This gives us the initial lookup results in the specified type. If we set a breakpoint here, run the compiler, and expr instanceTy->dump(), we get output which indicates that we’re performing lookup on the type Optional<S>. Great, just as expected! Inspecting the results of this call indicates that the compiler finds the foo(_:), again, as expected.

Several hundred lines further down, we find the following lines:

  // If we're looking into a metatype for an unresolved member lookup, look
  // through optional types.

This comment describes exactly the behavior we’re looking for. Let’s see how the compiler determines whether to do this lookup:

  if (result.ViableCandidates.empty() &&
      baseObjTy->is<AnyMetatypeType>() &&
      constraintKind == ConstraintKind::UnresolvedValueMember) {

Bingo! In addition to checking that the constraint we’re performing lookup for is from an implicit member expression, and that the type we’re looking into is a metatype (remember: implicit member expressions only work for static members), we’re also checking whether there are already any results. Since lookup previously found Optional.foo(_:), we don’t proceed with lookup into S, and so our expression doesn’t type check.

Conclusion

I hope this gave some insight into approaches that you can use to attack bugs found in the Swift compiler. I eventually fixed this issue with this PR, but explaining the fix is a topic for another post…