How to use Android CTS(Compatibility Tes Suite) and Monkey ToolTable of Contents 1. CTS Introduction----------------------------------------------------------------------------- 22. Before using CTS----------------------------------------------------------------------------- 33. Run CTS---------------------------------------------------------------------------------------- 54. Test log and test result----------------------------------------------------------------------- 95. CTS Verifier------------------------------------------------------------------------------------ 106. Set up CtsVerifier.apk------------------------------------------------------------------------ 107. Run CTS Verifier------------------------------------------------------------------------------ 118. Monkey Introduction------------------------------------------------------------------------- 139. Basic Use of the Monkey--------------------------------------------------------------------- 1310. Monkey option usage------------------------------------------------------------------------- 14 CTS IntroductionThe CTS runs on a desktop machine and executes test cases directly on attached devices or an emulator. The CTS is a set of unit tests designed to be integrated into the daily workflow (such as via a continuous build system) of the engineers building a device. Its intent is to reveal incompatibilities early on, and ensure that the software remains compatible throughout the development process.The CTS is an automated testing harness runs on your desktop machine and manages test execution. Individual test cases are executed on attached mobile devices or on an emulator. The test cases are written in Java as JUnit tests and packaged as Android .apk files to run on the actual device target.Work flow Chart : Before using CTS 1. Check out the dedicated testing PC(8168) , hereinafter called "PC8168". A: Check out that the Linux OS is workable by running the VMware player in PC8168 desktop. B: Click on the Ubuntu1104 to run Linux. Password:1qaz2wsx
C: Make sure the USB driver is ready. By using USB cable to connect PC8168 and a device(Here we use Nexus S as a reference phone), and tap "Turn on USB storage" button on Nexus S.
The original USB driver: D:\Android\usb_driver_r04-windows_for android phone, if needed. 2. Set up CtsDelegatingAccessibilityService.apk to a device
A: Connect the device to the PC8168 and make sure no any problem shows up. B: This apk is in D:\Android\android-cts-2.3_r3-x86\android-cts\repository\testcases\ C: Copy this apk to D:\android-sdk-windows\android-sdk-windows\platform-tools\ D: Go to D:\android-sdk-windows\android-sdk-windows\platform-tools\ directory E: Run the command: adb install -r CtsDelegatingAccessibilityService.apk3. Set up the device under test.
A. Settings->Application->Development, "USB debugging","Stay awake" and "Allow mock locations" three functions. B. Settings->Accessibility, "Accessibility" and "Delegating AccessibilityServices" functions. C. Settings->Display->Screen timeout, "30 minutes." or "Never Timeout" function. D. Settings->Location & security->Set up screen lock, "None" function. E. Make sure the device has a SD card plugged in and the card is empty. F. Do a factory data reset on the device. G. Make sure the device is at the home screen at the start of CTS(Press the home button). Run CTS 1.Tap Terminal function icon on PC8168 on Linux desktop. 2.Get familiar with basic Linux command : ~$ls and ~$cd A: ~$ls list files and directories. B: ~$cd <directory> jump to specific directory C: ~$cd .. go to previous directory D: sudo ./startcts start CTS environment.Note: sudo means "super user do" startcts : a shell script program. 3. Check out the default available CTS test plan , use command: ls --plan 4. Default CTS Test Plans: There are 8 default CTS Test Plans. They are "Performance, VM, Signature, Java, RefApp, AppSecurity, Android and CTS".
5.Start a CTS test plan ,for example, use command: start --plan VM
6.Start a CTS test plan ,for example, use command: start --plan RefApp 7.Start a CTS test plan ,for example, use command: start --plan AppSecurity 8. Other Test Plans follow the similar command line to test. Test log and test resultGet the test log and test result: take the AppSecurity test plan for example(start --plan AppSecurity) Test log: Use absolute path in the device, go to /home/stevenlee/Android/android-cts/repository, there is a log file named log_2011.06.28_16.00.45_.txt
Test result: Use absolute path in the device , go to /home/stevenlee/Android/android-cts/repository/results, there are a folder named 2011.06.28_16.02.51 and a zip file named 2011.06.28_16.02.51.zip, test result is included in the folder.
When you are ready, you can submit the report generated by the CTS to [email protected]. The report is a .zip archived file that contains XML results and supplemental information such as screen captures. CTS VerifierDevice implementations MUST correctly execute all applicable cases in the CTS Verifier. The CTS Verifier is included with the Compatibility Test Suite, and is intended to be run by a human operator to test functionality that cannot be tested by an automated system, such as correct functioning of a camera and sensors.The CTS Verifier has tests for many kinds of hardware, including some hardware that is optional. Device implementations MUST pass all tests for hardware which they possess; for instance, if a device possesses an accelerometer, it MUST correctly execute the Accelerometer test case in the CTS Verifier. Test cases for features noted as optional by this Compatibility Definition Document MAY be skipped or omitted.Every device and every build MUST correctly run the CTS Verifier, as noted above. However, since many builds are very similar, device implementers are not expected to explicitly run the CTS Verifier on builds that differ only in trivial ways. Specifically, device implementations that differ from an implementation that has passed the CTS Verfier only by the set of included locales, branding, etc. MAY omit the CTS Verifier test.
Set up CtsVerifier.apk (Let your device connect to PC8168 correctly through USB before setting up this .apk) 1. It exists in D:\Android\android-cts-2.3_r3-x86\android-cts\repository\testcases\CtsVerifier.apk 2. Copy this apk to D:\android-sdk-windows\android-sdk-windows\platform-tools\ 3. Go to D:\android-sdk-windows\android-sdk-windows\platform-tools\ directory. 4. Run the command: adb install -r CtsVerifier.apk, it will be installed automatically.
Run CTS VerifierRun CTS Verifier application Four main items under test
Audio Quality test Audio Quality test items Hardware test summary Hardware test summary
SUID File Scanner Checking Result
Accelerometer Test Shake device to test CTS Verifier P/F items
Monkey IntroductionThe Monkey is a program that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. You can use the Monkey to stress-test applications that you are developing, in a random yet repeatable manner.You can launch the Monkey using a command line on your development machine or from a script. Because the Monkey runs in the emulator/device environment, you must launch it from a shell in that environment. You can do this by prefacing adb shell to each command, or by entering the shell and entering Monkey commands directly. Basic Use of the MonkeyYou can launch the Monkey using a window command line on your development or from a script. Because the Monkey runs in the device environment, you must launch it form a shell in that environment, make sure that your window OS installed Android SDK.In PC 8168, C:\Program Files\Android\android-sdk\android-sdk-windows\platform-tools is the location where Android be installed, adding the SDK path into system environment path variable, so that we can run the monkey command anywhere in shell. Use: c:\adb shell monkey -help to check all of the monkey services. The basic syntax to run Monkey is : adb shell monkey [options] <event-count> Monkey option usage The Monkey is a command-line tool that that you can run on any emulator instance or on a device. It sends a pseudo-random stream of user events into the system, which acts as a stress test on the application software you are developing. The Monkey includes a number of options, but they break down into four primary categories:• Basic configuration options, such as setting the number of events to attempt. C:\adb shell monkey -v 5000 • Operational constraints, such as restricting the test to a single package. C:\adb shell monkey -p com.android.launcher -v 500 • Event types and frequencies. C:\adb shell monkey -p com.android.launcher --pct-touch 20 --pct-motion 20 --pct-anyevent 50 --pct-filp 10 --throttle 100 -v 500 • Debugging options C:\adb shell monkey -p com.android.camera --ignore-crashes -v 500|tee log.txt
How to use Android CTS(Compatibility Tes Suite) and Monkey ToolTable of Contents
1. CTS Introduction----------------------------------------------------------------------------- 22. Before using CTS----------------------------------------------------------------------------- 33. Run CTS---------------------------------------------------------------------------------------- 54. Test log and test result----------------------------------------------------------------------- 95. CTS Verifier------------------------------------------------------------------------------------ 106. Set up CtsVerifier.apk------------------------------------------------------------------------ 107. Run CTS Verifier------------------------------------------------------------------------------ 118. Monkey Introduction------------------------------------------------------------------------- 139. Basic Use of the Monkey--------------------------------------------------------------------- 1310. Monkey option usage------------------------------------------------------------------------- 14
CTS IntroductionThe CTS runs on a desktop machine and executes test cases directly on attached devices or an emulator. The CTS is a set of unit tests designed to be integrated into the daily workflow (such as via a continuous build system) of the engineers building a device. Its intent is to reveal incompatibilities early on, and ensure that the software remains compatible throughout the development process.The CTS is an automated testing harness runs on your desktop machine and manages test execution. Individual test cases are executed on attached mobile devices or on an emulator. The test cases are written in Java as JUnit tests and packaged as Android .apk files to run on the actual device target.Work flow Chart :
Before using CTS
1. Check out the dedicated testing PC(8168) , hereinafter called "PC8168".
A: Check out that the Linux OS is workable by running the VMware player in PC8168 desktop.
B: Click on the Ubuntu1104 to run Linux. Password:1qaz2wsx
C: Make sure the USB driver is ready. By using USB cable to connect PC8168 and a
device(Here we use Nexus S as a reference phone), and tap "Turn on USB storage"
button on Nexus S.
The original USB driver: D:\Android\usb_driver_r04-windows_for android phone, if needed.
2. Set up CtsDelegatingAccessibilityService.apk to a device
A: Connect the device to the PC8168 and make sure no any problem shows up.
B: This apk is in D:\Android\android-cts-2.3_r3-x86\android-cts\repository\testcases\
C: Copy this apk to D:\android-sdk-windows\android-sdk-windows\platform-tools\
D: Go to D:\android-sdk-windows\android-sdk-windows\platform-tools\ directory
E: Run the command: adb install -r CtsDelegatingAccessibilityService.apk3. Set up the device under test.
A. Settings->Application->Development, "USB debugging","Stay awake" and "Allow mock
locations" three functions.
B. Settings->Accessibility, "Accessibility" and "Delegating AccessibilityServices" functions.
C. Settings->Display->Screen timeout, "30 minutes." or "Never Timeout" function.
D. Settings->Location & security->Set up screen lock, "None" function.
E. Make sure the device has a SD card plugged in and the card is empty.
F. Do a factory data reset on the device.
G. Make sure the device is at the home screen at the start of CTS(Press the home button).
Run CTS 1.Tap Terminal function icon on PC8168 on Linux desktop.
2.Get familiar with basic Linux command : ~$ls and ~$cd
A: ~$ls list files and directories.
B: ~$cd <directory> jump to specific directory
C: ~$cd .. go to previous directory
D: sudo ./startcts start CTS environment.Note:
sudo means "super user do"
startcts : a shell script program.
3. Check out the default available CTS test plan , use command: ls --plan 4. Default CTS Test Plans:
There are 8 default CTS Test Plans. They are "Performance, VM, Signature, Java, RefApp, AppSecurity, Android and CTS".
5.Start a CTS test plan ,for example, use command: start --plan VM
6.Start a CTS test plan ,for example, use command: start --plan RefApp 7.Start a CTS test plan ,for example, use command: start --plan AppSecurity
8. Other Test Plans follow the similar command line to test. Test log and test resultGet the test log and test result: take the AppSecurity test plan for example(start --plan AppSecurity)
Test log: Use absolute path in the device, go to /home/stevenlee/Android/android-cts/repository, there is a log file named log_2011.06.28_16.00.45_.txt
Test result: Use absolute path in the device , go to /home/stevenlee/Android/android-cts/repository/results, there are a folder named 2011.06.28_16.02.51 and a zip file named 2011.06.28_16.02.51.zip, test result is included in the folder.
When you are ready, you can submit the report generated by the CTS to [email protected]. The report is a .zip archived file that contains XML results and supplemental information such as screen captures.
CTS VerifierDevice implementations MUST correctly execute all applicable cases in the CTS Verifier. The CTS Verifier is included with the Compatibility Test Suite, and is intended to be run by a human operator to test functionality that cannot be tested by an automated system, such as correct functioning of a camera and sensors.The CTS Verifier has tests for many kinds of hardware, including some hardware that is optional. Device implementations MUST pass all tests for hardware which they possess; for instance, if a device possesses an accelerometer, it MUST correctly execute the Accelerometer test case in the CTS Verifier. Test cases for features noted as optional by this Compatibility Definition Document MAY be skipped or omitted.Every device and every build MUST correctly run the CTS Verifier, as noted above. However, since many builds are very similar, device implementers are not expected to explicitly run the CTS Verifier on builds that differ only in trivial ways. Specifically, device implementations that differ from an implementation that has passed the CTS Verfier only by the set of included locales, branding, etc. MAY omit the CTS Verifier test.
Set up CtsVerifier.apk (Let your device connect to PC8168 correctly through USB before setting up this .apk)
1. It exists in D:\Android\android-cts-2.3_r3-x86\android-cts\repository\testcases\CtsVerifier.apk
2. Copy this apk to D:\android-sdk-windows\android-sdk-windows\platform-tools\
3. Go to D:\android-sdk-windows\android-sdk-windows\platform-tools\ directory.
4. Run the command: adb install -r CtsVerifier.apk, it will be installed automatically.
Run CTS VerifierRun CTS Verifier application Four main items under test
Audio Quality test Audio Quality test items
Hardware test summary Hardware test summary
SUID File Scanner Checking Result
Accelerometer Test Shake device to test CTS Verifier P/F items
Monkey IntroductionThe Monkey is a program that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. You can use the Monkey to stress-test applications that you are developing, in a random yet repeatable manner.You can launch the Monkey using a command line on your development machine or from a script. Because the Monkey runs in the emulator/device environment, you must launch it from a shell in that environment. You can do this by prefacing adb shell to each command, or by entering the shell and entering Monkey commands directly. Basic Use of the MonkeyYou can launch the Monkey using a window command line on your development or from a script. Because the Monkey runs in the device environment, you must launch it form a shell in that environment, make sure that your window OS installed Android SDK.In PC 8168, C:\Program Files\Android\android-sdk\android-sdk-windows\platform-tools is the location where Android be installed, adding the SDK path into system environment path variable, so that we can run the monkey command anywhere in shell. Use: c:\adb shell monkey -help to check all of the monkey services. The basic syntax to run Monkey is : adb shell monkey [options] <event-count>
Monkey option usage
The Monkey is a command-line tool that that you can run on any emulator instance or on a device. It sends a pseudo-random stream of user events into the system, which acts as a stress test on the application software you are developing.
The Monkey includes a number of options, but they break down into four primary categories:• Basic configuration options, such as setting the number of events to attempt.
C:\adb shell monkey -v 5000
• Operational constraints, such as restricting the test to a single package.
C:\adb shell monkey -p com.android.launcher -v 500
• Event types and frequencies.
C:\adb shell monkey -p com.android.launcher --pct-touch 20 --pct-motion 20 --pct-anyevent 50 --pct-filp 10 --throttle
100 -v 500
• Debugging options
C:\adb shell monkey -p com.android.camera --ignore-crashes -v 500|tee log.txt