The AQL Editor provides a framework to develop and test AQL files. Annotation Query Language (AQL) is the language in the InfoSphere® BigInsights™ Text Analytics component for building extractors. An extractor , sometimes referred to as annotator, is an information extraction program that extracts structured information from unstructured or semistructured text by using AQL constructs. AQL is the primary method of creating new extractors in InfoSphere BigInsights Text Analytics.
By using the AQL Editor, you can create an AQL file or script and develop the code to perform Text Analytics. An AQL script contains AQL statements. An extractor can consist of one or more AQL files and associated dictionaries and UDF JAR files.
You must compile AQL files before you run them. To ensure that the AQL compiler is started whenever you save an AQL file, select the Build Automatically option from the Project menu.
The AQL code of the extractor is compiled by the Text Analytics Compiler into Text Analytics modules (TAM files). A Text Analytics Module (TAM) file is a binary representation of the extractor and contains the execution plan of the extractor, the module metadata, and the compiled dictionaries. Non-modular AQL files are compiled into a genericModule.tam, and modular AQL files are compiled into a file called <moduleName>.tam
In its source form, an AQL module is a collection of AQL source files that are grouped under a folder whose name is identical to the name of the module. Optionally, a module can also contain dictionary files and binary UDF JAR files. A compiled AQL module contains the compiled operator graph, compiled dictionaries (if any), UDF JAR files (if any), and module metadata. AQL modules facilitate modularity and reusability of AQL code. AQL modules help build reusable text extractors that can be used as building blocks to compose complex Text Analytics applications.
The Problems view displays errors and warnings with your AQL code. If an error or warning occurs during compilation, it is displayed in the Problems view with detailed information. If you double-click the error or warning, the AQL Editor shows the specific line in the AQL file where it occurred. Errors and warnings are also identified on the marker bar of that particular AQL file in the AQL Editor.
For projects that use non-modular AQL code, if the error or warning occurs on the main AQL file or an included file that is present in the current project, the exact file and line number is highlighted. However, if it occurs in an included file that is not a part of the workspace or is a part of a different project in the workspace, then it is highlighted in the main AQL file.
For projects that use modular AQL code, errors and warnings in an AQL file are marked in the file at the lines where it occurred. However, in certain cases it might not contain associated file information. An error or warning of this type is marked against the project that contains the AQL code that is being compiled. The message contains the exact details and line number of the problem file.
| Column | Definition |
|---|---|
| Description | The compiler error description and suggestions for resolving the error. |
| Resource | The AQL file that contains the error. |
| Path | The path of the AQL file that contains the error. |
| Location | The line number of the error. |
| Type | The type of error that occurred. |
The Problems view also provides AQL compiler warnings for slow regular expressions. These warnings identify complex regular expressions that cannot be run with the InfoSphere BigInsights Text Analytics accelerated regular expression evaluation engine (SimpleRegex) because the regular expression involves advanced constructs, such as look-ahead, look-behind, and boundary matches. Such regular expressions are evaluated with the generic Java engine that can be much slower.
In certain cases, it might be possible to simplify the regular expression to avoid the use of such advanced constructs altogether, or replace their usage with other AQL constructs with equivalent semantics. As a result, the simplified regular expression may become compatible with the accelerated engine resulting in higher extractor throughput. For details, see regular expression tips in Follow the basic AQL guidelines.
Line 1: import view MyView from module sampleModule;
Line 2: create view PhoneNumber as
Line 3: extract regex /\d+(\.\d+)?/ on R.text as match
Line 4: from Document R;
Line 5: output view sampleModule.MyView;
Line 6: export view PhoneNumber;
If you place the cursor anywhere inside the token PhoneNumber in Line 6 and click F3, the action takes you to the definition of the view PhoneNumber in Line 2. Similarly, if you place the cursor anywhere inside the token MyView in Line 1 or 5 and click F3, the action opens the AQL file that contains the definition of MyView and takes you to the line number in the sampleModule module where MyView is defined. Similarly, you can navigate to the definition of any AQL identifier by using F3 navigation across modules and projects.
There is no AQL source in project's build path that contains definition of this element.
The Content Assist feature provides context-sensitive assistance to complete the current AQL statement. For example, when you type the statement output view in the AQL editor and call the Content Assist feature, the tool displays all of the views that can be output at the current cursor location. You can then choose a view from the list to complete the AQL statement.
| Token type | Color |
|---|---|
| Comments | Maroon |
| Strings | Green |
| Keywords | Navy blue |
| Variable type | Purple |
| Regular expression | Pink |
| Default | Black |
The Regular Expression Generator wizard helps you to create and refine a regular expression through a list of examples of the concept. Right-click a regular expression between the slash (/) characters in your AQL file, and select Regular Expression Generator to open the Regular Expression Generator wizard. In the wizard, you can generate or refine a regular expression. Click Finish, and the new expression replaces the old one. For more information, see Generating regular expressions.
The Regular Expression Builder helps you to build a regular expression by selecting constructs and combining them. Right-click a regular expression in your AQL file, and select Regular Expression Builder. The Regular Expression Builder opens with the regular expression as a parameter. Modify this regular expression by using the listed constructs. Click Finish, and the new expression replaces the old one. For more information, see Building regular expressions.
When you right-click the members in the tree, you can open the definition of that view in the AQL editor if the source of the view definition is available.
To start the AQL Dependency Hierarchy or Reference Hierarchy, select a view in the AQL editor, and select either Open AQL Dependency Hierarchy or Open AQL Reference Hierarchy. You can start the AQL Dependency Hierarchy or Reference Hierarchy only on views that are defined in the source directory of a project.