Graphical password systems based upon the recall and reproduction of visual patterns (e.g. as seen on the Google Android platform) are assumed to have desirable usability and memorability properties. However, there are no empirical studies that explore whether this is actually the case on an everyday basis. In this paper, we present the results of a real world user study across 21 days that was conducted to gather such insight; we compared the performance of Android-like patterns to personal identification numbers (PIN), both on smartphones, in a field study. The quantitative results indicate that PIN outperforms the pattern lock when comparing input speed and error rates. However, the qualitative results suggest that users tend to accept this and are still in favor of the pattern lock to a certain extent. For instance, it was rated better in terms of ease-of-use, feedback and likeability. Most interestingly, even though the pattern lock does not provide any undo or cancel functionality, it was rated significantly better than PIN in terms of error recovery; this provides insight into the relationship between error prevention and error recovery in user authentication.