Id: developer_guide.tex 15616 2009-10-26 08:56:43Z guelton
You can get a printable version of this document on
http://www.cri.ensmp.fr/pips/developer_guide.htdoc/developer_guide.pdf
and a HTML version on http://www.cri.ensmp.fr/pips/developer_guide.htdoc.
Warning if you use the HTML version and do some copy/paste” it has been noticed
that some characters are changes, such as the ’ that should be a simple plain quote
but is changed in something else...
The legacy WWW site is at http://www.cri.ensmp.fr/pips/ and a new one is at https://freia.enstb.org/pips .
This document aims at presenting PIPS development environment. It is not linearly organized: PIPS is made of several, sometimes interdependent, components. This paper thus begins with a presentation of PIPS directories. Then the shell environment is described. The next two chapters are devoted to two external tools on which PIPS relies: NewGen and the Linear library. Section 6 will then present PIPS make file policy. Sections 10 and 15 are devoted to PIPS libraries and passes. The next section briefly describes some conventions usually respected when developing in PIPS. The last two sections describe the bug policy of PIPS and some save and restore information.
This manual is not exhaustive. You can add your own sections and update existing ones if you find missing or erroneous information.
The reader is supposed to be a PIPS user [?], and to have read the reports about NewGen [?, ?]. A good understanding of pipsmake mechanisms would also be helpful.
The sources of NewGen, Linear and Pips are managed under subversion (svn). They are accessible from anywhere in the world by the http protocol at http://svn.cri.ensmp.fr/pips.html
There are 5 repositories for the various files:
There is also private validation directory on another server.
The subversion repositories are organised in standard subdirectories as advised for best subversion practices:
Moreover the pips repository includes
A crude script setup_pips.sh on the web site and in pips/trunk/makes allows to download polylib, newgen, linear and pips and to build a local installation of the softwares. For instance developer calvin can do the following to setup its own PIPS environment:
Before you enjoy PIPS, do not worry about the many error messages that are displayed: the header files are being recomputed from the C files. After a while, C files should be compiled without errors.
In order to rebuild only the PIPS infrastructure, just type make into the $PIPS_ROOT directory. If something has gone wrong, for instance because a software component is missing or outdated, it is advised to restart from scratch by runing make unbuild before trying again.
For more information about the PIPS Makefile infrastructure, see Section 3.4.4.
It is important to realize that:
To help digging into the sources, Doxygen is used to generate an interactive on-line version of the sources that can be seen at https://freia.enstb.org/pips/source. The call and caller graphs are useful to figure out what are the functions used, and more subtly what are the functions that call a given function (if you need to verify a function is correctly used or you want to change a function name).
As an experimented user, you may need to generate them again. The doxygen make target is propagated downwards down to the various doxygen directories in the products (NewGen, Linear and PIPS).
To push them on the WWW, you can use a make doxygen-publish, if you have the right access.
The make target doxygen-plain or doxygen-graph can be used to generate only a documentation without or with call and caller graphs. Note that generating the graph version for PIPS lasts several hours on a 2009 computer...
Of course, developers in PIPS should use the Doxygen syntax to have a more useful documentation.
Modules and submodules in Doxygen are quite useful to group related objects and structure concepts in the documentation. For an example, have a look to @defgroup, @addtogroup, @{ and @} in newgen/src/genC/genClib.c and the resulting module NewGen quick and intelligent recursion on objects (visitor pattern) and its submodules in the documentation. Be careful: a title group must be on the same line and cannot be split !
A good practice should be to present main useful functions and concepts to know by a PIPS programmer as modules and submodules.
There are many modes in Emacs that can help developments in PIPS.
To compile the sources you can use the menu Tools/Compile.... It presents errors in red and if you click on them you jump to the sources at the right location.
The check-compilation-errors-on-the-fly is quite fun (it is equivalent in programming to the on-the-fly spell checker flyspell-mode). PIPS makefiles support it. Try it with M-x flymake-mode.
The speedbar mode helps with a navigation window. More descrete, to have access from the menu to functions and other symbols in the local source file, test the imenu-add-to-menubar Emacs function.
You can use tags to index source files as explained in section 3.3.
Most PIPS contributors use Emacs to develop in PIPS, so some goodies have been added to ease those developers.
Have a look for example at section 8 for debugging. You should use the gud-tooltip-modes to display the value of an object under the mouse. Setting the gdb-many-windows to t transform Emacs (from version 22) in an interesting graphical IDE. You can select a frame by just clicking it and so on.
Most documentation (like the one you are reading) is written in LaTeX and some parts even use literate programming with a LATEX base, so the AUCTeX, PreviewLaTeX and RefTeX mode are quite useful.
Many macros and functions are available from the Linear and NewGen libraries, but also in PIPS. Their source code can be retrieved using the tags indexing mechanism under editors such as emacs or vi.
So the PIPS makefiles can generate tags index files to find the definition of symbols in the source files with your favorite editor and it is highly recommended to use the Emacs or VI editors to benefit from the tags. For example in Emacs, you can find, by typing a M-. on an identifier, the location where it is defined, M-, allows to find another location. Note that completion with TAB works with tag searching too. With vi, you can use C-] to jump on a definition of an identifier.
PIPS can produce tags files for VI by invoking make CTAGS or tags for Emacs by invoking make TAGS in the PIPS prod top directory and it will produce the tags files into the linear, newgen and pips subdirectories.
To build the tags for both VI and Emacs, just use a simple make tags. The tags files at the PIPS top directories need to be remade regularly to have an up to date version of the index. should be used for vi users.
You can also build specifically the tags for a development version of PIPS for example by making these tags explicitly in the newgen/trunk, pips/trunk and linear/trunk directories.
Then you need to configure your editor to use these tags files, for example in Emacs by using the customization of the Tags Table List with the Options/Customize Emacs menu or more directly in you .emacs with stuff like:
A basic understanding of subversion operations and concepts such as repository, checkout, status, commit are necessary to develop in PIPS. See for instance the SVN book at http://svnbook.red-bean.com/ and other resources on the Internet. This section assumes that you have a recent version of svn, version 1.5 or above, as these versions greatly improve the management of branches.
The development policy within the PIPS-related software is that the trunk version should hold a production quality software which should pass all non-regression tests that run nightly at CRI and elsewhere. Thus day-to-day development should not occur within this part, but in independent per-developer branches which should be validated before being merged back into the trunk.
If you are doing something non trivial or do not have commit rights on pips’ trunk, you shoud develop in a branch of your own. To do so:
For instance, a login calvin and some password may be attributed. Calvin will have his own development area under branches/calvin in the subversion repository.
The developer branch is private by default. It means that what you do in the branch is your own business, and no message is sent to the world on commits. For interns, the default policy is to send a message to the intern and the developer(s) responsible for his/her work.
For HPC-Project related accounts, branch commits are send to the corresponding list.
Please do commit your work at least once a day, even if it does not work yet!
Branches may also be used for linear and newgen development, in which case the local directory name for the checkout should be called linear_dev or newgen_dev respectively.
It creates a new branch as a working copy, which is then committed to the repository.
The idea is to branch a whole PIPS copy since it ensures that the developments you do in many places in PIPS (even at some place you would not have expected when you created the branch!) can be committed atomically.
CAUTION! take care to create branches only for the trunk. Creating branches for a subdirectory breaks svn bookkeeping.
CAUTION! When testing, check that the PIPS_ROOT environment variable points to your development branch instead of the trunk. You may also check for the right settings for EXTERN_ROOT NEWGEN_ROOT and LINEAR_ROOT variables, as well as your PATH to ensure that the right executable is used. Have a look to § 4 for more information.
With svn 1.6 it will be possible to type only:
If you change your initial idea, you can always revert back to your original version if it was committed before with a:
Well, it does not remove the new files that may be created previously in your local copy by the merge procedure, it only delete their meta-data from the subversion point of view. So the cleaner way is to merge into a new working copy of the trunk so that if things go wrong you can simply delete the working copy.
To know about where you are compared to the trunk, you can try a:
CAUTION! Merging must be done at the branch level which must correspond to the trunk. DO NOT merge in a subdirectory as it breaks svn bookkeeping for subsequent merges.
CAUTION! it is important to merge at the trunk level. If the merging is done on a subdirectory, it breaks svn merge bookkeeping used my the automatic mergings and reintegration because svn:mergeinfo properties are added on these subdirectories: when set, they must be manually removed in order to fix.
Do not forget that if you or someone else merges your branch into the trunk, things must be compiled with an up-to-date copy of all the PIPS environment and validated first against the last global PIPS validation to avoid havoc for the other PIPS developers.
Here is a typical behaviour you should follow:
For more information about the validation process, have a look to § 7. Since your validation mileage may vary according to your target architecture because of bugs or limitations (little or big Endian, native 32-bit or 64 bit architecture...) you should try to validate on various architectures.
The pips_branch script creates a full branch or tag of the same name in all 5 repositories (makefiles, 3 softwares, validation), and a bundle entry to extract all these files at once. The script needs full write permissions to all five repositories, thus it is restricted to core developers. The help option provides help.
To create a foo/bla named branch from trunk try:
It shows the various svn commands which are going to be issued. If you are happy with them, do that again with the -D option to actually run them. The created bundle can be extracted with:
Note that the -R options allows to fix the revision number to consider from each repository, instead of using the head revision.
This section is to be skipped at first reading and is only of interest for people wanting to modify more globally the PIPS infrastructure.
The compilation directories are managed by some global variables initialized as described in section 4.
If you compile into a directory, in the prod directory installed by the installation script, your branch directory or whatever, the compiled stuff from a directory goes to the corresponding bin lib directories next to the src directory. The Makefile definitions are taken from the corresponding makes directory next to the src sources.
For example, if you have a local PIPS copy in the svn/pips/trunk and you compile with make, it will build against $NEWGEN_ROOT and $LINEAR_ROOT, it will install into bin and will use Makefile stuff from makes. So if you want to modify the Makefile infrastructure, modify the makes directory and commit. It will be committed into the svn/nlpmake/trunk repository anyway.
The SVN centralized development model assumes that you are connected to the repository. If you are stuck in a train between Paris and Brest (in the very far west), you may find it inconvenient not to be able to access the repository history and other details about previous commits, especially if the current version is broken. In order to help preventing this situation anyway, PIPS is checked every minute and possibly fully recompiled.
So if you are Ronan, consider using git-svn or svk to have full copy of your branch or of the trunk on your laptop. Then you can develop offline including branching and commits. You will be able to push back into the SVN server your changes once you get back to civilized surroundings.
Why not switch all this stuff to git, which is so cool? There are several reasons:
At HPC Project we use GIT with central servers, so the previous points of view from CRI are rather out-of-focus...
In order to access pips logs at anytime, you may consider keeping a local copy of pips repositories anywhere you want, including on a laptop, with the svnsync command. The operation is mostly network bound: it takes under 15 minutes to copy the whole history of the 5 repositories from the CRI LAN. Beware that this copy must only be used read-only.
This section assumes that you have a good working knowledge of both GIT and SVN. If you use GIT, do not forget to save your repository regularly.
git-svn is a porcelain which ensures a limited compatibility between a centralized SVN repository and a local GIT repository. It provides the ability to pull/push commits from a single branch in an SVN repository. Beware of the limitations:
So just use git-svn to manage an already created branch which will be seen as the master branch by GIT. Creating new branches under GIT and hoping to have them appearing under SVN seems hopeless.
Here is a summary of operations3 :
Since in the PIPS repositories there is also some quite old history (based on RCS and SCCS4 ), the history does not fit into classical SVN trunk, branches and tags and thus cannot either be understood well by git-svn. So for ultimate PIPS historian which does not want to use svnsync, you can have also a copy of the repository with git-svn without using the trunk/branches/tags convention. For this, just use a plain git-svn with:
To have GIT ignoring the same files than SVN, you should try in each GIT top-directory a
How to organize this? For example Ronan Keryell uses this hierarchy in a PIPS directory:
To apply a same git command on GIT repositories below the current directory, you can use the pips_git script. By default, the action is to do the equivalent of a svn update, that is a git svn rebase:
because it is the most used command: it fetches new versions from the SVN repository and rebase your current work according to these last versions.
For example to get the status of all your GIT repositories, use:
There are also PIPS specific instructions to pips_git:
Many environment variables may be used by PIPS executables and utilities. They mainly describe PIPS directory hierarchy, compilation options, helper programs, and so on. PIPS should compile and may be launched without any such variables, as defaults are provided and computed at run time depending on the driver or executable path.
These variables should be initialized by sourcing the pipsrc.sh file for sh ksh bash shells, or pipsrc.csh file for csh tcsh shells. An initial version of these files is created by the setup script, and can be customized as desired.
Two variables are of special interest: $PIPS_ROOT which stores the root of PIPS current version, and $PIPS_ARCH.
The variable holds the current architecture. If not set, a default is automatically derived with the arch.sh script.
A PIPS architecture is to be understood not as strictly as a computer architecture. It simply a set of tools (compilers, linkers, but also compiler options…) to be used for compiling PIPS. Thus you can have several pips architectures that compile and run on a very same machine.
This set of tools is defined in the corresponding $PIPS_ARCH.mk makefile, where usual CC, CFLAGS and so make macros are defined.
This configuration file is automatically included by all makefiles, hence changing this variable results in different compilers and options to be used when compiling or running pips. Object files, libraries and binaries related to different pips architecture cannot be mixed and overwritten one by the other: they are stored in a subdirectory depending on the architecture, à la PVM. Thus pips versions are always compiled and linked with libraries compiled for the same architecture.
Here are examples of pips architectures:
Several variables are used to explain where to build PIPS components and from where the components are picked. It is useful when working with several working copy and branches at the same time.
There is also a ROOT variable that can be set to precise the target main directory for the compilation.
For example, if you want to compile an experimental version of NewGen and compile it and install into the main directory of the NewGen installation (selected by NEWGEN_ROOT) to compile a global PIPS version without changing its compilation recipe.
to compile the local version into the standard NewGen directory.
It avoids changing around NEWGEN_ROOT when recompiling PIPS with various versions of NewGen, Linear or even part of PISP (libraries in Libs...).
Another way to achieve the same behaviour could be done with:
to compile the local version into the local directory.
You just need to keep track of what you are doing, especially when working with various versions and validating...
This section describes the PIPS directory hierarchy. When extracting a repository with the setup_pips.sh script, three subprojects are extracted: newgen (software engineering tool, Section 8), linear (mathematical library, Section 14) and pips (the project). The three projects are organized in a similar way, with a makes directory with makefiles and a src directory for the sources. Other directories bin share runtime doc are generated on compilation (make compile or make build).
This directory contains the current version, namely pips subversion trunk directory. The $PIPS_ROOT environment variable should point to this directory. You can find the following subdirectories:
PIPS makefiles, and some helper scripts.
The PIPS Sources are here.
Having a copy of this subtree and of the makes directory is enough for fully recompiling PIPS, including the documentation, configuration files and various scripts used for running and developping the PIPS sofware.
The newgen sub-directory contains the description (in LATEX) of the internal data structures used in the project, as they are used in the development and production hierarchies. The local makefile transforms the *.tex files describing PIPS data structures into NewGen and header files, and exports them to the include directory. Thus the documentation actually is the sources for the data structures.
The wpips-epips-user-manual sub-directory contains the user manual.
The pipsmake sub-directory contains the definition of the dependences between the different interprocedural analyses as a LATEX file, from which are derived the pipsmake.rc configuration file used by PIPS at runtime.
Pips current binaries that can be executed.
These binaries include pips (the main program), tpips (the same program with a interactive interface based on the GNU readline library) and wpips (the window interface).
Non architecture-dependent executables needed or used by pips are also available, such as old Init Select Perform Display Pips pips shell interfaces, and pips tpips wpips epips jepips launchers.
Actual architecture-dependent binaries are stored in different sub-directories depending on $PIPS_ARCH which describes the current architecture, as discussed in Section 4.
Configuration files, such as pipsmake settings.
Also some configuration files, automatically generated from the LATEX documentation in src/Documentation. Important files are pipsmake.rc and wpips.rc.
They include shell (sh and csh), sed, awk, perl and lisp scripts.
Pips documentation, namely many PDF files describing various aspects of PIPS, the internal data structures and so on. Important documents: pipsmake-rc (the dependence rules between pips interprocedural analyses), developer_guide (this document!), ri (the description of the pips Intermediate Representation, also known as the Abstract Syntax Tree (ast)).
This directory populated from the various src directories and is also used to build the static PIPS home page.
This HTML documentation of PIPS is populated from the various src directories. There are real HTML files, and some generated from LATEX files.
This directory is also used to build the static PIPS home page.
Environments needed for executing pips-compiled files. Namely hpfc and wp65 use a PVM-based runtime and also xPOMP the graphical user programming interface. This directory includes the files needed for executing the generated files (as header and make files or compiled libraries), but not necessarily the corresponding sources.
Other architecture independent tools and files used for developing pips (as opposed to running it), such as validating PIPS, dealing with the SVN organization.
But there are also some scripts used to help PIPS usage, such as displaying some graphs or dealing with PIPS output.
This directory contains all shared files needed for building PIPS. It includes C header files automatically generated from NewGen specifications and for each library, and from some file in the documentation. Also pips architecture configuration file that define makefile macros are there.
This directory contains PIPS compiled libraries (lib*.a) ready to be linked. They are stored under their $PIPS_ARCH subdirectory.
The GNU make utility is extensively used to ensure the coherency of PIPS components. However, most Makefiles are not written by hand, but are automatically composed according components from nlpmake package and automatic dependence computation. It eases development a lot, by providing a homogeneous environment all over the PIPS software parts.
If you need some specific rules or targets, you have to write some makefile whith named ending with the .mk extension: they will be included in the global Makefile.
The rationale for relying on GNU make rather than make is that in the previous situation the software was relying heavily on SUN make special features, thus was highly not portable. Now it is more portable, say as much as the GNU softwares. The complex makefiles of PIPS rely on GNU extensions such as conditionals for instance.
Global targets are targets useful at the top of PIPS to with global meaning such as building a global binary file from the sources.
Useful targets defined:
If you use a global compiling target in the prod directory, it will compile NewGen, Linear and PIPS to build coherent binaries.
If you you use a global compiling target in a sub-component such as NewGen, Linear or PIPS, it will be compiled and eventually linked with the global libraries (defined according to the environment variables, by default the prod directory). So it is useful to have a coherent PIPS compilation of a branch (such as in you pips_dev) that is linked with production NewGen and Linear.
Local targets are actions with a more local meaning to help developing some passes or documentation parts, such as:
There are other rules to describe…
Note that if you work on many libraries or documentation at the same time, to have a coherent compilation, you should not use local target but rather use some global targets from § 6.1 at a higher level in your branch or your working copy of the trunk.
THIS PARAGRAPH IS OBSOLETE.I cannot see how to compile without installing to the PROD directory... make libxxx.a just recompiles the local *.c files, and creates the local library file libxxx.a. This is useful when making a lot of changes to check the syntax; or when making changes in several directories at the same time: in this case, the proper policy is to a) chose one of these directories as the master directory, b) build the libxxx.a in the other directories, c) make symbolic links towards them in from the master directory, d) and finally link in this last directory. If your pips architecture is not ., don’t forget to link the proper libraries in the proper sub-directory.
A typical example of a local Makefile file is provided in Figure 1.
By default, PIPS make does not stop on error but reports them on standard error output. But if you want to have make stopping on the first error to help debugging, just add FWD_STOP_ON_ERROR=1 to the make invocation.
The validation is stored in a separate subversion repository http://svn.cri.ensmp.fr/svn/validation
For an overview of the validation daily use case, you should also have a look to § 3.4.2.
Different phases are validated in distinct sub-directories. Running the validation uses the pips_validate script.
See the README at the root of the trunk for details on how to run the validation. Basically, one must simply type make.
See various examples around to add new validation files. The preferred way is to provide a source file MyTest.f or MyTest.c, a corresponding MyTest.tpips script, and store the standard output of the .tpips script into MyTest.result/test.
During the validation process (when you execute a make validate), the validation output, wich is stored in MyTest.result/out, is compared against the reference MyTest.result/test to know if the validation is correct. So to install a new validation item, you need to create the reference MyTest.result/test. pips_validate -a (see afterwards) can help to install a new validation reference file and directory. Note that an additional transformation on output and/or reference file is possible for special cases. In fact depending on the compiler used for example, you can get floating point numbers in different formats. Then you might want to post process the output to make it compliant with the reference. To make this possible a filtering phase is included in the validation process and is explained in § 7.3.1.
For more complex validations, a .tpips script without a direct corresponding source file but that may use many files from various location is also possible.
CAUTION: use a relative and local path in the .tpips file to reference the source file.
To do more subtle things (nightly validation with mail notification, etc) it is possible to use validation scripts instead.
But it is best not using the following validation scripts directly, but rather to rely on the Makefile in the validation directory, which has very convenient targets including validate accept clean. Using this approach does not require the validation directory to be under $PIPS_ROOT.
The main user interface is from the Makefile at the validation top directory and is usable through few make targets:
The validation can also be restricted to some directories by setting the TARGET environment variable, such as with
The VOPT make variable can be used to pass and modify default options for the validation.
As an output the make validate creates a RESULTS directory. This directory contains: a SUMMARY file, and 3 different file types (.err, .diff and .out). Those files are named using the following rule : FolderName_TestCaseName.xxx
Three validation scripts are available:
The arguments are names of subdirectories in $PIPS_ROOT/../../validation or source files within these directories. See pips_validate -h for help and a list of options.
Tests are organized more or less on a library basis in validation. Non-regression tests are characterized by one or more source programs which must have a known result.
Set PIPSDBM_DEBUG_LEVEL to 5 to check data structures when they are stored and to 9 to check them when they are delivered; some data structures are not checked, among them, all that contain external types;
The validation is performed for each source file found in a directory. These validation scripts must not rely on absolute path, otherwise moving the validation around is impossible. As the tests are run in the validation directory which contains the source, relative paths always work.
The scripts looks for validation scripts in the following order:
Validating the error stream is a very bad idea as it leads to never ending regression variations. It is even worth when the output comes from external tools such as compilers error output;
First it performs some substitutions (tested_file, tested_dir, tested_name and TESTED_NAME are substituted by the full file name, the validation directory, the name prefix in lower or upper) and then it executes the generated test file;
The environment variables FILE and WSPACE contain the full-path file name and the file name prefix;
For maintenance purposes, the default_tpips approach is the best choice (only one source file to maintain for all the sources of a validation directory). However, this can be used only if the very same test is to be applied. As a second choice, the .tpips script is encouraged. It is very important not to validate stderr output and pips with some level of debugging on, because these outputs are not very stable from one architecture to another.
This script is also used by the various nightly validations such as pips_at_night.
If the script is used without any parameter all the validation acceptation is asked for all the validation differences.
If the script is called with --new, then the validation acceptance is asked against what is newer in the last validation than in the previous one.
Once you have accepts the new states as a reference for some validation items, you need to commit these changes to the validation repository, if and only if the corresponding pips sources are already in the subversion repository of course!
If you are Emacs oriented, setting the PIPS_GRAPHICAL_DIFF environment variable to pips_emacs_diff to use graphical Ediff merge mode sounds like a good idea. You can use it like:
The script accepts two types of argument:
As described previously, for all those entries the previous validation will be made the reference for future validation tests. Note that you need to run the validation first.
Once you have run the pips_force_accept script, you need to commit these changes to the validation repository, if and only if the corresponding pips sources are already in the subversion repository of course!
Once the tpips script has been run, the current output has to be compared with the expected one (the reference). As explained previously, before to perfom a basic difference (using the diff shell command) the pips_validate script search for user defined filters to apply on the outputs. This search is done as follow:
A filter has to be a shell script getting an input file as its first parameter and printing the filtered file to the standard output.
Several issues have been encountered in the past with validation scripts.
First of all, validation scripts and related test cases are defined by the developper, not by the maintainer. The developper knows what the test case is about, and he/she fails to document the goal either in the source code or in the script... because he/she is in a hurry. Sometimes, the name of the test case can help the maintenance, but this is largely untrue for large librairies like Transformations. So, please, comment the goal(s) of your test cases.
By the way, is it a good idea to have many goals for one test case? It’s attractive for the deelopper when many different cases and combination of cases must be tested. But it’s a nightmare for the maintainer when one difference shows up or when PIPS core dumps if no unit test cases also exist because he will have to isolate the buggy part from the whole test case. So unit test cases are much eaasier to maintain, even though larger test cases are also very useful to check interactions and robustness.
Secondly, most test cases are not supposed to contain user errors. So, please, set the property PIPS_ABORT_ON_USER_ERROR to TRUE.
Thirdly, should test cases be restricted to test the output of the corresponding library or should they also test the outcome of the local results on other libraries? It is tempting to be strict and to select the first option, but, most of the time, some of the internal output will not appear in a text form after some prettyprinting. The consequences are only visible in the output of a later phase or transformation. So it is allowed to use any output in any of the library validation suites. For instance, the validation of a parser is not restricted to the display of PARSED_CODE.
As a consequence, multiple outputs may be combined in the resulting test file. In case of regression bug, it is necessary to know what the components of the test file are. So, please, use echo liberally in your validation scripts to comment the output. This way, the maintainer will spot quickly which phase fails or generates a different output.
Fourthly, in the same way, when a transformation or an analysis is built on top of other transformations and/or analyses, it is useful to keep track of the intermediary results, each of them being duly commented. The origin of a change can be located much faster that way.
Fifthly, to ensure portability, do not create dependences on /usr/include as it varies from compiler and system version to version. If header files are necessary and if they must show in the output, please use a preprocessed version of your source code as input code. It can easily be retrived from he database in directory Tmp. Use %ALLFUNC rather than %FUNC to avoid unwelcome output. Note: using and displaying fixed header files is not compatible with executing the resulting code as this may cause portability problems.
Finally, the input source code and the output source code should be compiled and executed whenever it is possible. Care must be taken to use only files in the database, i.e. compile the input after the create command. Use unsplit to produce the output source and compile it in directory Src. Do not forget to remove the executable and the object files. Executable and object files should have unique names in the validation directory to let the validation process run in parallel when multicore machines are available.
PIPS is developped to handle legal source code only. It must nevertheless detect all kinds of errors and issue warnings. To validate PIPS behavior when errors occur is tricky because errors are printed out on stderr together with system and version and time dependent information. Probably, an error log file should be used instead, but this is not implemented yet. Meanwhile, use the suffix .test to keep all PIPS output for the non-regression test. The case with warnings is easier because PIPS provides a Warnings file in the database. Do not forget to close the database and to explain what you are doing before you display this file.
Some test cases may become obsolete, but we do not have resources to maintain them properly.
Test cases can be factorized using default_tpips and/or default_test, but the same rules should be applied. Of course, it is less flexible to show information about a specific function among hundreds.
Scripting used to be shell based when PIPS was started. It is now mostly tpips based. Such scripts should end with a close, a delete of the current workspace5 , and a quit.
When multiple source files are used, it is better to keep them in a subdirectory. The maintenance of the validation library is easier and the output of the validation process is not clobbered with information about skipped files.
Scripts can sometimes look stupid because a quit appears in the middle. Do not forget that they are used to track bug and that additional information may be useful. For the same reason, the delete are often commented out. The cleaning is then performed by the higher-level validation process.
With the wide availability of multicore chips, the validation process could be parallelized. This implies that scripts remains as independent as possible of each other. They shoud not use file names such as a.out.
PIPS use a lot of macro processing to ease the programmer life with NewGen, but, in some cases, it may be difficult to understand what happens to the C source code. To help the programmer, there is a special +cpp target to build a CPP-preprocessed version of a file. For example
make a preprocessed version of the file ricedg.c. To help debugging, gcc-specific options are added to have the macro definitions at the beginning of the file too, to keep the comments and display the list of included files.
We use environment variables xxxx_DEBUG_LEVEL to control debugging output at the library level. So, one library (or phase) x calls library y, the entry points in x should handle the debugging level as Y_DEBUG_LEVEL and not X_DEBUG_LEVEL. See for instance the main entries in library pipsdbm. You can detecte them because they all contain a call to debug_on() on entry and a call to debug_off() on exit.
A library with different components may also define several debug levels, with different names. For instance, SAC_DEBUG_LEVEL, SAC_ATOMIZER_DEBUG_LEVEL, SAC_SIMD_LOOP_UNROLL_DEBUG_LEVEL, etc... Unspaghettify is supposed to be controlled by CONTROL_DEBUG_LEVEL, or by something more restrictive.
Let suppose you develop your own CUDA library. Then you should have your own CUDA_DEBUG_LEVEL, used with debug_on("CUDA_DEBUG_LEVEL"), debug_off(), ifdebug() to guard debugging code and pips_debug() to reduce the number of fprintf(stderr,...) and add useful information automatically, e.g. the current function name. Another debugging macro, debug, is now obsolete and must be replaced by pips_debug which add the current function name automatically. So, the second parameter of calls to debug, which is the current function name, must be deleted. This makes code maintenance easier.
The current debug level is a global variable called the_current_debug_level, which can be set easily under gdb. Calls to debug_on and debug_off push and pop the hidden debug level stack.
If the environement variable is not set or if it set to 0, there is no debug output. The largest value for debugging is 9.
Another macro, pips_assert, is useful to enforce contracts between callers and callees. The first argument describes the assertion, but it is often misused to give the current function name. This is useless because the information is added by the macro itself.
NewGen is a software engineering tool, from which almost all PIPS data structures are generated. It is described in ???. It is considered as an external tool, and is independent from the PIPS project. However, it is required for building pips from scratch, and the Newgen runtime library (genC) must be linked to pips.
NewGen sources are in the directory $NEWGEN_DIR. The global organization is similar to the PIPS organization.
$NEWGEN_ROOT and $NEWGEN_ARCH refer to the current version and architecture to be used. If $NEWGEN_ARCH is not defined, is defaults to $PIPS_ARCH and then to “.”, so that if one develop or update NewGen as part of PIPS there is no trouble.
It may (alas!) sometimes be useful to consult them. Several functionalities are useful when debugging:
Activation:
gen_debug |= GEN_DBG_CHECK;
Desactivation:
gen_debug &= ~GEN_DBG_CHECK;
And when all else fails:
gen_debug = GEN_DBG_TRAV_OBJECT
print obj->_type_
p Domains[obj->_type_].name
For example, picked from the RI:
p e->_entity_name_
and since a label is an entity as declared with:
you can get the information you want with:
print stmt->_statement_label_->_entity_name_
Fabien: NewGen should be simplified to use stmt->label->name
It should be also possible to use PIPS and NewGen macros (accessors and so on) from GDB instead of these NewGen internals, if the sources are compiled with the good options (-ggdb -g3). You may remove the -O option from the CFLAGS in the makefiles to avoid variable optimizations into registers. A good and simple practice to achieve this is to use:
In the future, gdb functions compatible with Newgen generated C macros will be provided6 .
To ease this kind of inspection, some keyboard accelerators can be defined inside the Emacs debugger mode by adding into your .emacs such as:
These function are then available with the C-x C-a binding, so you click on an interesting variable and then use
There are tools quite useful for tracking memory issues that may happen when dealing with complex structures in C as in PIPS.
The more powerful is Rational Purify, but unfortunately it is non free. But since it is a good tool, that makes sense to invest in such a tool. It works by instrumenting all the memory access at link time. The execution is slower.
Valgrind is another tool to run a program in a virtual machine that monitor every bit of the memory to detect memory corruption or the use of an uninitialized bit (yes, not byte) of memory. Of course the execution is quite slower. Interesting use is:
but other options are described in the manual, especially in the section MEMCHECK OPTIONS. You can put the options in your ~/.valgrindrc or the VALGRIND_OPTS environment variable. For a power user, some interesting options to begin with:
Electric fence (libefence) also proves to be usefull when you need to stop exactly when an illegal memory operation happens. Note that because of regcomp, the EF_ALLOW_MALLOC_0 environment variable has to be defined when debugging tpips.
Another solution is to ask the compiler to instrument the code at compile time to detect memory corruption later at run time. This is done with gcc by compiling with -fmudflap option (or -fmudflapth when PIPS is multi-threaded...). The run-time behaviour is controlled at execution with the MUDFLAP_OPTIONS environment variable.
The various documentations, including the source of the web pages, are stored in the src/Documentation directory.
For example this guide is stored in src/Documentation/dev_guide.
Have a look also on § 3.1 about Doxygen documentation of the sources.
The various PIPS documentations can be published on the web site http://www.cri.ensmp.fr/pips/ with the pips_publish_www once you have compiled it.
To publish the Doxygen documentation of the sources, see § 3.1, once you have compiled the Doxygen version.
There is also a new dynamic web site https://freia.enstb.org/pips that reference better the information stored onhttp://www.cri.ensmp.fr/pips/ . The idea is to have the content edited naturally and collaboratively onto https://freia.enstb.org/pips and from time to time injecting back the content into the http://www.cri.ensmp.fr/pips/ by editing the src/Documentation/web directory.
The source files for the library L are in the directory $PIPS_ROOT/src/Libs/L .
A Makefile must be created in this directory. It can be copied from an existing library since it ends with PIPS special include instructions used to simplify the make infrastructure, but it must be updated with the names of the local source and object files. Additionnal rules compatible with the make syntax can also be added in this Makefile, but now the preferred method is to put them in local.mk as explained later. For local lex and yacc files, use the $(SCAN) and $(PARSE) macros, and do not forget to change the ’yy’ prefixes through some sed script or better with the correct parser generator options to avoid name clashes when linking PIPS (which already includes several parsers and lexers).
The Makefile calls the PIPS make infrastructure to define for example a default recompile entry to install the library in the production hierarchy, typically in directories include and lib at the root.
Great care must be taken when creating the library header file L.h. This file includes the local header file L-local.h (which contains macros, type definitions…) written by the programmer and the external functions declared in the local source files, which are automatically extracted using cproto.
L.h must never be directly modified as indicated by the warning at the beginning. It is automatically generated when invoking make header, or when L-local.h has been modified, but not when the source files are modified. This avoids inopportune recompilations when tuning the library, but can lead to coherency problems.
If a main program to test or use the library exists, then it is necessarily a pass (see section 15), or a main(). But the easiest way to test a library is to test through the PIPS infrastructure after declaring it as explained in section .
PIPS data structures are managed by NewGen. For each data structure declaration file in $PIPS_ROOT/src/Documentation/newgen, NewGen derives a header file that defines the object interfaces, which is placed in $PIPS_ROOT/include.
For instance, the internal representation (i.e. the abstract syntax tree) is called ri7 . It is described in the LATEX file ri.tex in $PIPS_ROOT/src/Documentation/newgen. It is then automatically transformed into a NewGen file ri.newgen. And finally, NewGen produces an internal description file ri.spec, a C file with the object method definitions and a header file ri.h, which must be included in each C file using the internal representation.
Thus, it is not possible to build a new library with the name ri, because the corresponding header file would be called ri.h. The library in which higher order functions concerning the ri are placed, is rather called ri-util. It should be the case for all the other sets of data structures specified using NewGen. Such existing libraries are text-util and paf-util. However, mainly for historical reasons, there are numerous exceptions. Many functions are in fact in the library where they were first necessary. An example is the syntax library (the parser). It should be refactored out some day.
Library dependence cycles must be avoided. Links are not easy to manage when a module a from library A calls a module b form library B which itself calls a module a′ from library A. This situation also reflects a bad logical organization of libraries. Therefore, PIPS libraries must be organized as a DAG, i.e. they must have a partial order. With modern linkers, it is no longer an issue but we still have the issue at the .h level if we have cross-dependencies such as cross-recursion.
Several utilities, such as analyze_libraries, order_libraries, grep_libraries, order_libraries_from_include (see $PIPS_ROOT/src/Scripts/dev), can be used to determine the dependences between libraries, a total order compatible with the library partial order, and the possible cycles. The results are stored in /tmp/libs.? where the question mark stands for several values:
Some libraries implement functionalities which are accessible by the users via the pipsmake mechanism. For a good comprehension of the material contained in this subsection, the reader is referred to the corresponding documentation.
A PIPS phase MYPHASE is implemented as a C function named myphase with a single argument, which is the name of the module (a function or procedure) to analyze. It returns a boolean value, which indicates whether the computation has performed well. It therefore resembles the following dummy function:
This phase is located either in a new file8 in an existing library or in a new library named myphase. The latter is not compulsory, but it is advised for new contributors. If the new phase requires global variables to tune its behavior, the properties mechanism must be used. The new phase must acquire the necessary resources by invoking pipsdbm. Recursively invoking pipsmake is forbidden at this level.
This section does not deal with the case where a new kind of resource is introduced. This is the object of Section 10.4 instead.
Here are now the steps to install a new library named mylib with a new C file named myphase.
Skip this step if you are using an already existing library.
Note that the cproto command is used to generate the library header file mylib.h. To force this file to be build again, type make header.
Since we use svn to deal with the code, it is a good practice to add svn:ignore properties to ignore considering automatically generated files, such as for Libs/gpu:
If you need to introduce new environment variables or to modify old ones (not likely):
Note that the pipsrc.*sh files created at setup time by the setup command is not affected, you must copy them again by hand since they were copied from an older version at download time. So this change will only affects future installations. Another possibility to synchronize with the production version is to download a new version of the setup_pips.sh script (or better make a link to MYPIPS/prod/pips/makes/setup_pips.sh for example to have an always up-to-date version) and run again setup_pips.sh. It won’t erase the directories but will pull the new version of subversion and will reconstruct the pipsrc.*sh files.
For instance, for a phase myphase which reads a resource res1 and produces a resource res2 for the current module, the rule and its alias are:
You can document your new phase using the PipsPass environment. It takes the phase name as a mandatory argument. Its body briefly document the phase.
If you need to further refer to this phase, use \PipsPassRef{myphase}.
Note that all phases generally use the program entities, for example if you display some code in debug routines..., that is all the objects from the current C or Fortran program and from C and Fortran run-times that have a name. This resource, the global symbol table, is named PROGRAM.entities.
where propname is the name of your property, default_value is a boolean, an integer or a string ((TRUE—FALSE), [0-9]+, "[^"]*"). In the second case, add a \PipsPropRef{propname} reference to the used property.
Execute the command make recompile to recompile the pipsmake library.
New properties (that are global variables with a user interface in PIPS) may be necessary for the fine control of the new phase. They must be declared in the pipsmake-rc.tex file.
Execute the command make recompile to recompile and install updated versions of pips, tpips and wpips.
With fast computers, it may be easier to skip the previous steps and to rebuild PIPS from directory PIPS_ROOT using make build.
It can also be necessary to update the shell scripts Init, Build, Select, Perform, and Display in directory src/Scripts/drivers.Are they still in use since tpips? Should be automatically generated anyway... RK
A new resource is declared by its use or its production by one of the rules in the file pipsmake-rc.tex (see below). Here are the steps to perform to declare a new resource myres once the phase myphase from the library mylib has been declared and installed as shown in the previous section. This assumes that the function
is available.
For instance, for a phase myphase which reads a resource another_res and produces a resource my res for the current module, the rule and its alias are:
The new resource (if any) must be declared to the resource manager pipsdbm:
For file resources, the pipsdbm data usually is a string representing the file name within the pips database. All such resources are suffixed with _FILE. It is declared as:
Then follow with the end of section 10.3.4 and then section 10.3.6.
It is necessary to recompile PIPS before any test, because changing the header files generated from pipsmake-rc.tex can lead to inconsistencies which only appear at run time (the database cannot be created for instance).
NewGen data strutures for PIPS are defined in LATEX files which are in the directory $PIPS_ROOT/src/Documentation/newgen. These files include both the NewGen specifications, and the comments in LATEX format. DDL10 files are automatically generated from the previous LATEX files. When declaring a new data structure, two cases may arise:
This file has to be declared in the local Makefile (in $PIPS_ROOT/src/Documentation/newgen) for its further installation in the PIPS hierarchy. It merely consists in adding its name mysd.tex to the F.tex variable list.
If the data structure is to be managed by pipsdbm, i.e. it is declared as a resource in pipsmake-rc.tex, then
must be updated.
It is not compulsory to immediately execute the command make in these directories. But it will be necessary before any global recompilation of PIPS (see below).
Then, in both cases, the next steps have to be performed:
Global variables should be avoided, or at least carefully documented to avoid disturbing already existing programs. Properties (variables with a user interface) may be used instead.
If new functionalities are required, it is better to keep the existing module as such, and to create a new one, with another name.
For global variables, initialization, access and reset routines must be provided. Do not forget that there may be several successive requests with wpips or tpips, whereas tests performs with shell interfaces (for instance Build) are much simpler.
That means that the work depends on the objectives and the complexity of the new language. If the Pascal language is added it should be easy, if an object-oriented is added, it should be hard…
NewGen iterators are very effective to visit the PIPS internal representation, as any other NewGen objects. For more documentation, look at the Doxygen module NewGen quick and intelligent recursion on objects (visitor pattern) and its submodules.
Considering a call to:
it will walk on all statements (because of statement_domain) included in the tree-like structure some_statement.
Its structure is more or less (from the module statement point of view):
some_filter will be called in a depth-first fashion (prefix-order) with the visited statement given as parameter, so that some_filter can investigate iteratively every statement. If some_filter returns FALSE, gen_recurse stop descending (but not visiting somewhere else).
That means here: first on the block statement, recurse into it, the call, the loop, recurse, the block, etc
So it recurses into a statement only if the return value of the filter is TRUE, otherwise it does not recurse.
Then on all statements previously visited, some_action is called, in a bottom-up fashion (postfix-order).
NewGen is the tool used in PIPS to describe the objects which PIPS deals with. NewGen generate data structures and methods to relief the programmer from gory details in a portable way.
There are many interesting features coming with NewGen:
The XML backend is called DOoM in NewGen because it implements many parts of the DOM (Document Object Model) in XML.
XML support is included in NewGen if it is compiled with the USE_NEWGEN_DOOM macro-constant defined. Then, the XML support is used if the NEWGEN_DOOM_WRITE_XML environment variable is set to 1.
MAY BE OBSOLETE DOWN FROM HERE…
This directory contains the version of PIPS currently being developed. It is a mirror of the organization under $PIPS_ROOT/src. Each library is developed and tested under this subtree, linking by default with the installed libraries in $PIPS_ROOT/Lib.
Thus new versions can be developed without interfering with other developments. Once a new version of a library (or pass, or script, or anything) is okay and validated, it is installed under the $PIPS_ROOT/Src subtree and becomes the reference for all other developers who may use its services.
When developing or debugging several related libraries at the same time, you can used the development version of these libraries by using Unix links (ln -s) to one of the directory under the corresponding $PIPS_ARCH subdirectory. When building a new binary, the linker takes these local libraries first.
At CRI, this directory is a large piece of disk space to be used temporarily to analyze large programs and to core dump large pips processes; no permanent, non-deductible information should be stored there.
The linear library is also independent from PIPS. Its development was funded by the French organization CNRS.
Its root directory is /projects/C3/Linear/ ($LINEAR_DIR). This directory is further divided into similarily to Pips and Newgen. Thus there are Production ($LINEAR_ROOT) and Development directories. $LINEAR_ARCH (which defaults to $PIPS_ARCH and . (dot)) can be used for building the library.
The source files for a pass p can be found in the directory Development/Passes/p. A pass, as opposed to a library, corresponds to an executable, which can be used by the users.
TODO : update this section since config.makefile does no longer exist !
In this directory, a local config.makefile must be created. It must be updated with the names of the source files. Then the local Makefile can be automatically derived using pips-makemake -p. This Makefile contains among others an install entry to install the pass in the Production hierarchy.
The libraries used to build the pass are in the directories Production/Libs and Externals. The corresponding header files are in Production/Include and Externals.
To maximize code reuse, it is recommended to limit code development at this level to functionalities really specific to passes. Everything else should be placed in libraries (for instance in top-level).
At the library level, it is also possible to create local executables by invoking make test, make ttest or make wtest.
If a pass and a library are simultaneously developed, misc and parallelize for instance, one of the library must be chosen for the link, parallelize for instance). It is then necessary to look for misc.h and libmisc.a in Development/Lib/misc instead of Production/Libs/misc. This can be done very simply in the config.makefile file by giving the names of the libraries to use:
libraries are named libXXX.a where XXX is the logical name of the library: vecteur, prettyprint, misc,etc.
It is theoretically unuseful to put libraries in the Makefile dependencies, because header files which are automatically generated are in these dependencies, and are systematically modified each time an installation is performed. However, this does not work if the pass p calls the library a which needs the library b, and if p does not directly needs the library b: modifications to b will not provoke the link of p.
Each important library uses an environment variable XXX_DEBUG_LEVEL to control debugging messages. This rule hase an exception: PARSER_DEBUG_LEVEL corresponds to the syntax library .
Level 0 corresponds to the normal behaviour. The highest level is 8. Great care has to be brought to calls to debug_on() and debug_off(), because successive debug levels are stored in a stack, and it could disturb its coherency. The debug of a function of another library should not unconsciously be activated.
When a bug has been detected, a small Fortran program (bug.f for instance) must be written to reproduce the bug.
If the library xxx is responsible for this bug, move bug.f in Tests/Bugs/xxx/. This responsibility can be found using XXX_DEBUG_LEVEL or a debugging tool.
Then record the new bug in $PIPS_DOCDIR/pips-bugs.f.tex.
Go to the directory /Tests/Bugs/xxx. In order to use the executable of the library xxx, create a symbolic link towards it (ln -s Development/Lib/xxx/pips for instance). Here are now the different steps to perform:
and verify that the bug has been eliminated.
Once these operations have been done, the program bug.f and the results obtained for its parallelization must be added to the non-regresson tests in the directory /Tests/Validation/Xxx. For that purpose, you can run the command bug-to-validate bug.f. But beware that this command is a script shell which provides a mere parallelization of bug.f. To transfer all the programs from the directory Tests/Bugs/xxx to /Tests/Validation/Xxx, you can use the command dir-to-validate. Again, this is very convenient when the desired test is the default test (see Section 17.3).
Remark: When several libraries are responsible for the bug, say xxx1,…, xxxk, chose one of the libraries to build the executable by linking with the other libraries. Once the problems are fixed, do not forget to run make install in all the libraries.
TODO : update this section to the valid svn directory.
These directories /Tests/Validation/Xxx contain the non-regression tests for each significant library xxx, as well as demonstration programs, benchmarks, ….
For each Fortran file bug.f, there exists a test file bug.test, and a sub-directory bug.result in which the results are stored in a test file. All these files can be generated by bug-to-validate bug.f when dealing with a bug in Tests/Bugs/xxx. If this command has been used, the reuslt directory contains the results of a mere parallelization. For more complex results, a specific test file must be written, as well as a script inspired from bug-to-validate to install the necessary files in the validation directory.
If the same test file is valid for a whole directory (ex : Validation/Flint), a generic file default_test can be created in this directory. In this file, the generic names for the program are tested_file and TESTED_FILE. Here is the default_test script of Validation/Flint:
Notice that if there exists a local test for the file (bug.test), it will be used first. The priority starts from local files to general ones.
The command Validate can be used to measure the impact of a modification by comparing the new behavior of PIPS with the preceding one.
To use it to check the parallelization process, put your test file, say mytest.f which contains the main program MAIN and a subroutine SUB, into one of the directories in ~pips/Pips/Tests/Validation directory. You can also create your own directory there if you want to ensure that a particular aspect of PIPS behaves the correct way.
Once mytest.f is in such a directory, say kludge, you should do the following thing.
Check that the output in the MODULE1, MODULE2, ... files is what you want ... and that’s it!
After a while, if you want to check that PIPS still does that it was supposed to do, go into Validation and type
If there is any problem, you will get a mail message that tells you want the problem is. You can also type Validate to check everything.
To print a nice version of PIPS output, you can use commands like tgrind:
All the source of PIPS can be pretty-printed with Doxygen and are browsable. For example, see http://doxygen.pips.enstb.org.
Errors which are the most difficult to find generally are dynamic allocation errors: The allocated space is accesssed after being freed (dangling pointer), or a zone larger than the allocated zone is referenced (too long copy of a character string for instance).
A first checking level is available with a set of SUN primitives (man malloc.h), by linking the program with /usr/lib/debug/malloc.o. The choice of the dynamic allocation library can be made in pipsrc.ref, or in the curretn environment, but without any guarantee for the link.
A second level, a lot more efficient, but also much slower, is offerd by a public domain library, malloclib-pl11. This library systematically overwrites freed zones with a special pattern, and checks that string operations are valid. It also records in which files allocations and desallocations are performed.
To use it efficiently, all the source files which dynamically allocate or free memory must include malloc.h with double quotes (and not < > signs). This is achieved in most cases because NewGen includes malloc.h. But some files, such as character strings manipulation files, do not include GenC.h: #include "malloc.h" must be added to them.
PIPS must then be entirely recompiled (see section 2.2), as well as the libraries in $PIPS_EXTEDIR, after renaming (mv) dbmalloc.h into malloc.h in $PIPS_EXTEDIR and in the directories containing the sources of the external libraries (NewGen and Linear). Before linking, the environment variable PIPS_LIBS must have been modified, either in the current environment, or in pipsrc.ref, to include the correct library.
It is recommended to keep an version of PIPS linkeed with a reasonnably fast library to be able to create large databases for the tests. To avoid compiling and linking too many things each time, it is also recommended to modify the shell variables $PIPS_BINDIR and $PIPS_LIBDIR to keep both versions of PIPS.
config.makefile, 20
gen_consistent_p, 28
gen_debug, 28
gen_defined_p, 28
libraries.make, 33
make
depend, 20
fast, 20
full, 20
header, 31
pipsrc.csh, 34
pipsrc.sh, 34
ri, 31
Validate, 23, 24