#helpbutton | Explore Tumblr posts and blogs

abbeyjackson · 8 years ago

Text

A Guide to Xcode’s UITest

Originally written in December 2015 and published at: https://blog.metova.com/guide-xcode-ui-test/

Note: This article was written in mid December 2015. At that point UITest was still experiencing several bugs. Many times I had tests fail that the next day would pass after no changes. I have not used UITest since as my current project has a QA team that writes and performs UI and functional tests and so I do not know which if any of the issues I discovered in the “gotchyas” section below are still applicable. When looking at the docset keep in mind that UITest is for both iOS and OSX testing and therefore there may be OSX specific items mentioned (click vs tap for example).

If you find something in this article that is no longer true/relevant please comment or reach out to me somehow and let me know. Thanks!

At the time of writing this article there were no official docs. Apple now provides documentation: https://developer.apple.com/documentation/xctest/user_interface_tests

I have tried to update the links in this article to point to the new documentation however I may have missed some. When moving my articles onto this new site I have not edited them except to fix broken links or incorrect information. I like to preserve the original mood, bad grammar and all!

http://developer.mandarapte.com/files/2013/12/01-Xcode.png

Apple’s new UITest suite has some developers excited, and others disappointed in lost functionality. UITest works differently than the functional testing solutions developers have come to rely on, such as KIF. Instead of giving you access to elements themselves, UITest gives access to proxy elements with minimal parameters to interact with. Learning to separate unit testing and unit based uitesting from functional testing can be a frustrating experience, especially when working with a new framework that still has its own kinks to work out. It is my hope the following guide will help developers form some clarity into the methodology needed when working with UITest.

How It Works

UITest is unique in that it does not use the project code to test, it exists outside the app. UITest instead looks at what is available in the simulator and returns to us instances of XCUIElement based on what it finds, XCUIElement and XCUIApplication are the proxies that are used for this. A button is a XCUIElement of type XCUIElementType.Button (or just .Button), for example. This means you are not able to access ObjC or Swift classes/objects nor their associated properties. You can only access XCUIElement properties. This also means there is no @testable or other such imports as in XCTest.

UITest includes a new Record feature which can be used to record actions and help speed up the writing of tests. Tests can be written without the help of the Record feature, and therefore can be part of test-driven development, however when writing tests after writing the code you are testing it is best to first use Record and then modify the results. This is because it is sometimes difficult to predict how an element will be exposed to UITest until you are more familiar with it. It also helps to identify issues you may have been unaware of such as multiple items with the same identifier.

XCUIApplication is the proxy that is used for testing. It launches a new instance of the app for every test giving you a clean slate to run your test on. As with XCTest there are setup and teardown methods and you can encapsulate common functionality such as writing a function in the test file to clear a text field which you can then call in each subsequent test that is needed. You can also make an extension on XCUIElement to reuse code in any part of the test suite.

XCUIApplication launches and terminates the app (launch and terminate, respectively). You can also pass arguments to the application on launch using launchArgumentsand pass an environment using launchEnvironment. From the header files / docset: "Unlike NSTask, it is legal to modify the environment [and launch arguments] after the application has been launched. These changes will not affect the current launch session, but will take effect the next time the application is launched."

The XCUITest API

UITest is built on top of the XCTest Framework and includes new additions to the API:

Class Reference:

XCUIApplication, XCUICoordinate, XCUIDevice,

XCUIElement, XCUIElementQuery, XCUIRemote,

XCUISiriService, XCUIScreenshot, XCUIScreen

Protocol Reference:

XCUIElementAttributes, XCUIElementTypeQueryProvider

XCUIScreenshotProviding

Constant (Enum) Reference:

XCUIDevice.Button, XCUIElement.Type, XCUIApplication.State,

XCUIRemote.Button, XCUIUserInterface.SizeClass

Constant (Struct) Reference:

XCUIElement.KeyModifierFlags

Identifying Elements -> XCUIElementQuery

Elements are identified using queries, most often using a combination of identifiers and types. Elements are unique, in order to use an element the query must return one element. Apple has provided several shortcut queries (app.tables returns a XCUIElementQuery of all the tables and app.table["table identifier"] would return the XCUIElement of type .Table with the specified identifier vs the long form which would be app.containingType(.Table, identifier: "table identifier")) and one can use the long form or the short form depending on preference and need. Queries are chained together and variable names can be substituted throughout the chain.

let app = XCUIApplication() let cellQuery = app.tables.cells .containingType(.StaticText, identifier:"String Identifier")

let helpButton = cellQuery.buttons["?"] helpButton.tap()

There are often multiple ways to reach an element, for instance a cell with identifier "Purchase Cell" that contains a button "Buy Item" and also a label with identifier "Purchase" could be found several ways and if you use Record you should be presented with a clickable token that allows you to choose which version you would like to use:

let cell = XCUIApplication().tables.cells["Purchase Cell"]

let cell = XCUIApplication().tables.cells .containingType(.Button, identifier: "Buy Item")

let cell = XCUIApplication().tables.cells .containingType(.StaticText, identifier: "Purchase")

let cell = XCUIApplication().descendantsMatchingType(.Table).element .descendantsMatchingType(.Cell).matchingIdentifier("Purchase Cell")

If you were wanting to interact with the label or button rather than the cell you could do the following:

let buyButton = cell.buttons["Buy Item"]

let purchaseLabel = cell.staticTexts["Purchase"]

These are short forms, the long form would be:

let buyButton = cell.childrenMatchingType(.Button, identifier: "Buy Item")

Or if you did not have a way to identify the cell but you could identify the table you can also do this:

let buyButton = table.descendantsMatchingType(.Button, identifier: "Buy Item")

If you had a cell that had two buttons with the same identifier you would need to identify which button you would like by it's index value. Using Record is recommended as in practice I found the elements were not always at the expected indices. Note elementAtIndex will autocomplete but is already deprecated (as of Dec 2015), use elementBoundByIndex instead.

let button = XCUIApplication().tables.cells["Purchase Cell"] .buttons.elementBoundByIndex(1)

This would find the button at index 1 out of all buttons on the cell.

let button = XCUIApplication().tables.cells["Purchase Cell"] .buttons["Buy Item"].elementBoundByIndex(1)

This would find the button at index 1 out of all buttons on the cell with identifier "Buy Item".

If you have only one item of a certain type there is an additional shortcut, element. For example if you have a cell containing one button, that button can be accessed like so:

let button = cellQuery.buttons.element

For element identification there is additionally elementMatchingPredicate(NSPredicate) and elementMatchingType(XCUIElementType, identifier:String?). These are separate from

containingPredicate(NSPredicate) and containingType(XCUIElementType, identifier: String?) which are checking the element for items inside it whereas the elementMatching... options are checking the values on the element itself. Finding the correct element will often include combinations of several query attributes.

If you have difficulty interacting with an element, printing the accessibility hierarchy can help (print(app.debugDescription), and if that doesn't work here is a tip from shinobicontrols.com:

Sometimes when you tap on an element while recording, you’ll notice that the code produced doesn’t look quite right. This is usually because the element you are interacting with is not visible to Accessibility. To find out if this is the case, you can use XCode’s Accessibility Inspector. Once it is open, if you hit CMD+F7 and hover over an element with your mouse in the simulator, then you’ll see comprehensive information about the element underneath the cursor. This should give you a clue about why Accessibility can’t find your element.

https://www.shinobicontrols.com/blog/ios9-day-by-day-day2-ui-testing

Sharing Code Between App and UITest Targets

UITest is a completely separate entity from the app itself and therefore can only access UI elements however it may be beneficial to share code between the project and UITest target for writing your tests. To use code from the app target there are two options.

Add the file to Compile Sources (or check off UITest target in member dependencies on the file). Must remember to also compile all files used by this file whether or not the test will use them.

Take the shared code out of the app's files and into a new file and share that file between the two. This eliminates the need to also compile dependent files.

A good example of when this would be useful is given by Big Nerd Ranch where they show how you can use shared code to check if a to-do item is marked as finished (strikethrough). While we can not access the object's properties directly we can use shared code to return values that we can use for assertions.

Properties, Attributes, Methods & Queries

Please see either the Apple Documentation for a list of properties, attributes, methods and queries. I have deleted the information I collected about them from this article as I expect things have changed in the two years and it is best to learn about these things from the official docs.

Be aware that UITest uses the accessibility API to populate attributes for the XCUIElement under test.

How-To's and Gotchyas

This article was originally written in Dec 2015 when UITest was in it’s infancy. I am certain most of these gotchyas will have been addressed by Apple in Xcode updates since that time. If you work with UITest please let me know if any of these are no longer an issue so that I can remove them from this list.

Frames For Off-Screen Elements

Queries will return elements which are offscreen. If the frame is needed for test logic or an assertion, before getting the frame make sure it is on screen, usually by tapping on it first.

Asserting Adjusted Slider Values

adjustToNormalizedSliderPosition does not guarantee accuracy but rather is an estimate. I found setting 0.0 or 1.0 does give 0% and 100% respectively but otherwise it is best to XCTAssertNotEqual to the previous value rather than test the new value.

Inconsistent Test Failures

UITest is still buggy. Scrolling to a cell does not work consistently. In my tests I had it fail 20-30% of the time one day and then on another day not at all. I witnessed the same thing happen to another developer as well. I also experienced tests failing one day and passing the next, with no changes to the code. If you can not figure out why a test is failing I recommend moving onto another task and returning to debug at the end. When you come back you may discover it is a passing test after all.

Record Returns Wrong Element

UITest will sometimes grab an image instead of a button when tapping on a button. I also noticed that UITest prefers to return Static Texts (labels) rather than buttons, this may have to do with the view hierarchy and what element is on top.

Slider Will Not Move

Slider only slides using adjustToNormalizedSliderPosition if it starts at 0. A work around is to use pressForDuration:thenDragToElement and find an element nearby. Remember this may not be accurate unless you are dragging to an element at the extreme beginning (0%) or end (100%)

Multiple Matches Failure with Recorded Code

Sometimes Record does not identify elements correctly and a test will fail due to multiple matches or no matches when Record gives you code indicating there is one unique match. I found the multiple matches error happened most often when there were multiple matches to the query but the other elements were off screen at the time of Recording. The best way around this is to assign a unique identifier or to determine which index the element is at.

Multiple Matches Failure with Recorded or Written Code // How to Wait for Expected Result

Another reason for a multiple matches error is that queries will sometimes not be complete when the test moves onto the next line. A delay should be able to resolve this problem. We tell it to wait for exactly one result (self.count = 1), or wait for an element to exist or not exist (element.exists == true / false) before moving on and give it a timeout value. This is supposed to be fixed for Xcode 7.1.0 however users were again reporting the problem in 7.1.1.

let predicate = NSPredicate(format: "self.count = 1 || element.exists || etc") _ = self.expectationForPredicate( predicate, evaluatedWithObject: XCUIApplication().tables, handler: nil ) self.waitForExpectationsWithTimeout(5.0, handler: nil)

Recorded Code with Special Text/Characters Do Not Compile

Record also may not convert text properly. A button with a unicode identifier was extracted using “\U2192\Ufe0e" which results in a compiler error and had to be manually adjusted to "\u{2192}\u{fe0e}”. Be aware of this for all escaped characters and special texts such as those used in HTML Strings.

Record Fails To Capture Action

Record sometimes fails to capture actions. The best way to deal with this was to tap until an action was recorded, then do the action that you need and if it was recorded edit the test after to remove the unneeded actions. Several times I was not able to record an action for one element but was able to record it for the element next to it and then edit the test code to target the correct element.

Interacting With Elements Currently Off-Screen

You do not need to include scrolling actions to reach an element far down a page. Once the element is identified it can be interacted with no matter if it is on screen or not; the interaction should bring it on screen. When using Record scrolling actions will be recorded however these lines can be deleted from the test once the element itself is identified.

Work Arounds for Accessing Object Properties and States

As you can not access the object’s properties you can not for instance tell if a label is NSString or NSAttributedString (for example to check if text is strikethrough). This is because UITest only uses accessibility attributes and a person using voice-over would not be told text is strikethrough unless it is part of the object’s accessibility traits. Aside from using shared project code (as described above), a work around for this would be to change the item’s accessibilityLabel to be prefixed with “strikethrough” or if in a to-do list “done” would be more descriptive.

Sometimes you need to check that an element's state has changed. You can achieve this usually by checking element.value before and after an action. If it is a label or image you can use the accessibility attributes to have it treated like a button.

Nested Collection Views

UITest was unable to identify a UICollectionViewCell nested in a UICollectionView nested in a UITableViewCell. I was not able to test if a nested tableView has the same problem. It could see the element in the collectionView cell but not the cell itself so state changes on the cell could not be asserted.

Animations Causing UI Tests to Fail

In several cases, animation may cause the UI tests to fail (for example, waiting for a button to fade in and be clickable). In some cases, you may be able to wait until the button exists, but in some cases, it is better to disable your animations in your UI tests, and instead use unit testing to check for valid animations.

To disable animations in your test, add the following to your app delegate:

if NSProcessInfo.processInfo().environment["animations"] == "0" { UIView.setAnimationsEnabled(false) }

And this to your set up for your UI test:

let app = XCUIApplication() app.launchEnvironment = ["animations": "0"] app.launch()

Resources

https://developer.apple.com/videos/play/wwdc2015-406

https://www.bignerdranch.com/blog/ui-testing-in-xcode-7-part-1-ui-testing-gotchas/

https://www.shinobicontrols.com/blog/ios9-day-by-day-day2-ui-testing

http://masilotti.com/ui-testing-xcode-7/

http://masilotti.com/ui-testing-cheat-sheet/

https://medium.com/@larcus94/ui-testing-with-xcode-7-221d16bad276#.7uwbcrfsr

#iosTesting

0 notes