-
-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Performance Testing
To run performance tests for all pages:
python -m scripts.run_performance_tests
To run test for a specific page:
python -m scripts.run_performance_tests --test_name=page_test
page_test is the name of the file containing that test eg. splash_test. Refer to core/tests/performance_tests
directory for the files containing these tests.
The framework has two major components to it:
-
Performance Data Fetcher: We use selenium and browsermob-proxy to fetch performance data for the different Oppia pages. Selenium helps in programmatically interacting with a browser and browsermob-proxy is used to capture HTTP Archive (referred to as HAR) data by recording the communication between the client and server. The HTTP Archive format is a JSON-formatted archive file format used for logging a web browser's interaction with a site. It contains detailed performance data, including information about page loading and displaying and per-resource statistics. Each entry contains the URL requested and request and response headers. For additional details, please see: https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/HAR/Overview.html
-
Performance Metrics Provider: Refer to:
PageSessionMetrics
andMultiplePageSessionMetrics
objects inperf_domain.py
.
We consider the following metrics for the first time a user opens a page as well as for a return user (to test caching of various resources).
- Total page size.
- Total page load time.
- Dom ready time.
- Request time.
What to do if tests fails?
If it is something like:
self.page_metrics.get_page_load_time_millisecs(), 3000)
AssertionError: 3715 not less than or equal to 3000
This means that the threshold set for this metric is not met. This could happen due to:
- You introduced a change which alters the value of the metric. In this case, there is an increase in the total page load time.
- This could also happen due to variability in timings from machine to machine. Make sure that these tests are the only (major) active process running.
To write tests for a new Oppia page:
- Create a new file in
core/tests/performance_tests/
following similar name conventions to the existing test files there, i.e it should be something likepagename_test.py
. - This folder also contains
test_config.py
, config file for performance tests. Add a unique key for the new page and add a dictionary entry containing thresholds for different metrics to theTEST_DATA
object. Also, add correspondingpreload_options
. -
base.py
has some utility functions for tests which should be used to record page metrics. -
core/tests/performance_framework/perf_domain.py
has functions starting withget_metric_name
that give us the quantitative value for that metric. In case you require any other metric, add it to the same file. Also, refer to description regarding the raw stats that we have.
- Page session statistics.
- Page session load timings.
-
WebDriverException: Message: chrome not reachable
: Try setting bypass proxy for local addresses if you are behind a proxy server.
Have an idea for how to improve the wiki? Please help make our documentation better by following our instructions for contributing to the wiki.
Core documentation
Developing Oppia
- FAQs
- How to get help
- Getting started with the project
- How the codebase is organized
- Making your first PR
- Debugging
- Testing
- Codebase policies and processes
- Guidelines for launching new features
- Guidelines for making an urgent fix (hotfix)
- Testing jobs and other features on production
- Guidelines for Developers with Write Access to oppia/oppia
- Release schedule and other information
- Revert and Regression Policy
- Privacy aware programming
- Code review:
- Project organization:
- QA Testing:
- Design docs:
- Team-Specific Guides
- LaCE/CD:
- Developer Workflow:
Developer Reference
- Oppiabot
- Git cheat sheet
- Frontend
- Backend
- Backend Type Annotations
- Writing state migrations
- Calculating statistics
- Storage models
- Coding for speed in GAE
- Adding a new page
- Adding static assets
- Wipeout Implementation
- Notes on NDB Datastore transactions
- How to handle merging of change lists for exploration properties
- Instructions for editing roles or actions
- Protocol buffers
- Webpack
- Third-party libraries
- Extension frameworks
- Oppia-ml Extension
- Mobile development
- Performance testing
- Build process
- Best practices for leading Oppia teams
- Past Events