Skip to content

Configuration data for the Qt CI system. This repository contains various settings used by the test scripts running in Qt’s Continuous Integration system.

Notifications You must be signed in to change notification settings

qtqa/testconfig

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This repository contains much of the configuration which defines what Pulse
projects do and some Coin shared code. Below you will find documentation
only for the first one.


HOW IT WORKS:
=============

Each Pulse project is made up of several stages.  A stage is used to test
a particular configuration of the tested product.  Like this:

 Qt 4.7 Autotests
  |
  +-- linux-g++-32 Ubuntu 9.10
  |
  +-- win32-msvc2008 Windows XP
  |
  +-- macx-g++ cocoa OSX 10.6
  |
  ...etc

The combination of project & stage names are used to look up all the settings
for the current test.  Effectively, all Pulse projects and stages should
call exactly the same top-level script, and that script looks up the settings
for the test in a big hash, where the key is the combination of project and
stage name, and the value is the settings and script needed to do the test.


LAYOUT:
=======

Note that project and stage names have whitespace and special characters
replaced with an `_' when mapping to a filename.

 projects/<projectname>/properties

  This directory contains one file named after each Pulse property to set.
  During the test, the Pulse property named after the file will be set to
  the first line of content in the file (properties with values containing
  a newline character aren't supported).

  The values here may refer to other Pulse properties using the syntax
  $(my.property), and may refer to environment variables using the syntax
  ${MY_ENVIRONMENT_VARIABLE}.  If you
  want to include a literal $ or \ in the value, you must escape it with
  \ (i.e., use \$ or \\).  Having a cyclic dependency between properties
  is an error.

  These properties are global to all build stages for the given project.


 projects/<projectname>/environment

  This directory contains one file named after each environment variable to
  set.  During the test, the environment variable named after the file will
  be set to the first line of content in the file.

  The values here may refer to other Pulse properties in the same way
  as values in the `properties' directory.  Please note that environment
  variables can refer to Pulse properties, but not to other environment
  variables.

  These environment variables are global to all build stages for the given
  project.


 projects/<projectname>/repository

  This file contains a single line specifying an additional git repository
  needed to do the test. The format is:

   git/uri#gitbranch directory

  ...which will cause the specified git repository and branch to be checked
  out into the given directory.

  For Qt5, this will usually be:

   git://gitorious.org/qt/qtqa _qtqa_latest

  ... as most test scripts are located in this repository at time of writing.
  By convention, the destination directory for the clone starts with the _
  character, to differentiate it in build logs from directories contained in
  the source under test.

  This file is optional if no extra repo needs to be checked out for the test.


 projects/<projectname>/script

  This file contains a single line specifying the script to run in order to
  do the test.

  Please do not try to do anything tricky here. In particular, don't put any
  test logic here! Just run a single command stored _outside_ of this
  repository.

  This will usually be something like:

   perl _qtqa_latest/scripts/setup.pl --install && perl _qtqa_latest/scripts/<some_script>.pl


 projects/<projectname>/forcesuccess

  If this file exists and contains a true value, then test.pl will exit with
  a 0 exit code even when the test fails.

  This is intended to be used for "trial" stages which should be put into Pulse
  but may not yet be 100% reliable.  Using this, the stage can be put into the
  true production environment with minimal risk of causing build failures.  The
  forcesuccess file can later be deleted once the stage is thought to be stable.

  Note that `forcesuccess' is unable to catch certain classes of critical
  failure, such as a hanging build.


 projects/<projectname>/inherits
 projects/<projectname>/stages/<stagename>/inherits

  If present, these are symbolic links to another project or stage which the
  current project or stage inherits from.  All settings from the parent
  project/stage will apply, unless otherwise overriden.


 projects/<projectname>/stages/<stagename>/properties
 projects/<projectname>/stages/<stagename>/environment
 projects/<projectname>/stages/<stagename>/repository
 projects/<projectname>/stages/<stagename>/script
 projects/<projectname>/stages/<stagename>/forcesuccess

  These work identically to their counterparts in the project directory.
  When a file exists in both the project and stage directory, the version
  in the stage directory is preferred.  Therefore, one can set default
  values in the project directory and override them on a per-stage
  basis.


 manual_overrides/properties
 manual_overrides/environment

  These work identically to their counterparts in the project directories
  but they're applied only when running test.pl outside of Pulse.
  Use this sparingly! It's mainly a hack to prevent people from
  accidentally uploading binary packages, test results, etc.  Without
  this, one could easily clobber real test artifacts when testing locally.

  test.pl guesses that it's running outside of Pulse based on the
  absence of certain environment variables set by Pulse.


CAVEATS:
========

Adding projects/stages in this source tree does _not_ automatically
create the corresponding projects/stages in Pulse. That step needs
to be done manually.

There is no way to represent "required resources" here.  This means
that the part of the Pulse configuration specifying which machines
a stage can run on remains not under source control.  This can
be mitigated by reducing our usage of "required resources" as much
as possible.
In particular, requiring a resource should ideally _not_ set any
additional properties when doing a test.

There is (currently) no way to represent per-agent properties here.
This could be implemented if it seems worth the effort.

Since this repository uses symbolic links in some places, please be
very careful making any changes on Windows.  It is easy to accidentally
destroy the symbolic links when working on that operating system.

About

Configuration data for the Qt CI system. This repository contains various settings used by the test scripts running in Qt’s Continuous Integration system.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages