Customisation¶

Out of the box Atramhasis tries to make as few assumptions as possible about setup. We have taken care to ensure that significant parts of the application are easy to customise and expect most installations to have custom code. We’ve shipped Atramhasis with sane defaults so you can get a quick feel for the capabilities of the software. However, we do not advise running a production instance with only these default settings.

Creating your own project¶

Follow the README at atramhasis_scaffold_cookiecutter

This gives you a clean slate to start your customisations on.

Database¶

By default the scaffold comes with a simple SQLite database. This is more than enough for your first experiments and can even be used in production environment if your needs are modest. You can always instruct Atramhasis to use some other database engine, as long as SQLAlchemy supports it. Configure the sqlalchemy.url configuration option in development.ini to change the database. See the documentation of SQLAlchemy for more information about this connection url.

Database initialisation¶

To initialise the database, simply run the following.

# Create or update database based on
# the configuration in development.ini
$ alembic upgrade head

Custom alembic revisions¶

If you have a need to create your own tables, or do custom database changes we suggest you do so in another alembic branch next to the atramhasis branch.

First edit the alembic.ini file so it contains the following:

script_location = alembic
version_locations = %(here)s/alembic/versions atramhasis:alembic/versions

Second, initialise alembic in your project:

$ alembic init alembic

This will create an alembic folder for your own revisions.

To create your first revision, the command is a little longer:

$ alembic revision -m "first revision" --head=base --branch-label=myproject \
--version-path=alembic/versions

Note

if you need your alembic revisions to run after the atramhasis - for example if you want to create foreign keys to atramhasis tables - you can use --depends-on <hash> where the hash is the latest revision hash from atramhasis. This hash can be found by using alembic heads. In this example it is 184f1bbcb916

$ alembic heads
184f1bbcb916 (atramhasis) (head)

Having created a revision like above will have created a second alembic branch. Your alembic should have 2 heads now:

$ alembic heads
184f1bbcb916 (atramhasis) (effective head)
975228f4f18c (myproject) (head)

Adding additional revisions will look like:

alembic revision -m "second revision" --head=myproject@head

Warning

Not using a seperate branch will add revisions to the atramhasis alembic branch. While this may work initially, this may create split branches and multiple heads when upgrading atramhasis in the future and this is ill-advised

Whenever you would use alembic upgrade head to upgrade your database, you now have to use heads plural instead.

# Create or update database based on
# the configuration in development.ini
$ alembic upgrade heads

Running a local server¶

Your custom version of Atramhasis can now be run. Run the following command and point your browser to http://localhost:6543 to see the result.

$ pserve development.ini

Creating conceptschemes¶

Atramhasis is now running but does not contain any conceptschemes. A conceptscheme is a single set of concepts and collections, also known sometimes as a thesaurus or vocabulary. All concepts you describe in Atramhasis must be part of exactly one conceptscheme. Every conceptscheme is handled by a provider. The conceptscheme is the domain specific knowledge you are trying to describe, the provider contains some logic on how to display this knowledge and handle it. We have split the two for various, mostly technical, reasons. You might want to have staff with IT experience to create and maintain the providers, while the conceptschemes might be maintained by domain experts you do not want to burden with technical decisions. Atramhasis makes this possible.

Warning

Prior to version 2.0.0, creating conceptschemes and providers required manual interventions and writing a little bit of code. This has changes significantly. If you have an older installation, please make sure to read upgrading_providers before upgrading.

Adding a new provider through the UI¶

The easiest way to register a new provider is to use the UI. Surf to http://localhost:6543/admin. On your right you will see the menu where most of the interaction takes place. Bottom right, there’s a button labelled Providers. Pressing this button takes you into a menu that allows you to configure providers.

This dialog lists all providers currently registerd in your Atramhasis instance. Most of these will be regular SQLAlchemyProviders that will serve your conceptschemes and their concepts and collections. The dialog also lists other providers that cannot be managed through the UI. These are external providers that provide data from another source, such as the Getty Art and Architecture Thesaurus (AAT). See external_providers for more information on how to do this. For now it’s enough to understand there can be some providers that can’t be edited through the UI.

To add a new provider, press the Add a provider button. This will add a new provider and create the associated conceptscheme. Once the provider is created, the conceptscheme can be edited through the UI for conceptschemes. To create a provider the following information needs to be provided:

id: An identifier for the provider, can be a string or a number, eg. heritagetypes or 5. When left blank, an id will be assigned by Atramhasis equal to the DB identifier of the conceptscheme. The provider id is used in constructing URL’s for the application and will be visible in your final document URL.
conceptscheme uri: A URI for the conceptscheme. This should be a unique URI that resolves to the conceptscheme’s URL in Atramhasis. If you are creating a thesaurus of colours at http://data.me.org/colours, your id might be colours and the URI might be http://id.me.org/colours. This URI should redirect to http://data.me.org/colours. Another common option is to use http://data.me.org/colours#id as the URI for your conceptscheme.
uri pattern: A pattern that will be used by a UriGenerator to generate URI’s for new concepts and collections added to your conceptscheme. The pattern looks like a http URI with the marker %s as a placeholder for your concept or collection id. Continuing our example, this could be http://id.me.org/colours/%s or http://data.me.org/colours/%s#id.
default language: The default language of the provider. When a concept is requested, a single label is always returned for ease of display. Normally this is determined from browser settings or cookies, but if none of these are present, Atramhasis will try this language. Should generally be set to the main or prevalent language in your thesaurus. When not present, Atramhasis falls back to English as the lingua franca.
force display language: Even though the client can generally choose which language a concept or tree view should be displayed in, there are times when it’s handy to force the display language. This has been used before for biological taxonomies where you might always want to display the name in Latin, even though there’s a more local name available. See force_display_label_language for more information. For most thesauri this will be blank.
ID Generation Strategy: Every concept or collection needs a unique identifier. This identifier is used in generating URI’s. There are three possible strategies for generating ID’s:
- numeric (default): Before Atramhasis 2.0.0 this was the only option. It generates a numeric id based on the current highest id in the conceptscheme.
- guid: Generate a GUID as identifier for your concept or collection.
- manual: No identifier is generated by Atramhasis, but the user is required to generate them manually. Allows for full control over ID’s and URI’s, at the cost of a little more work.
subject: Add a certain tag to a provider. Currently there are two recognised tags:
- external: Must be added to an external provider, in code.
- hidden: Used to obfuscate a conceptscheme. The scheme will not be shown in the Atramhasis UI, except when the URL’s of concepts or collections are requested. This allows having a conceptscheme that can be used in applications, but is not the main focus of your Atramhasis instance. See hidden_providers for more information.
expand strategy: One of the features of Atramhasis is quickly determining all the narrower concepts of a certain concept, mainly to be used in querying other systems. There are two options:
- recurse: Use recursive queries to determine the narrower concepts. For large vocabularies this generally requires good caching.
- visit: Query a special visitation table that is pre-computed. Will be much more performant, but requires that the table is recomputed regulary.
When in doubt, use recurse and pay attention to caching.

After filling out the required fields and pressing Save, a provider and an associated conceptscheme will be created. The conceptscheme will be very barebones, so it’s recommended to add extra labels, notes and sources to the conceptscheme to inform end users.

Warning

The UriGenerator that you configure only generates URI’s when creating new concepts or collections. When importing existing vocabularies, please be sure to create the URI’s before or during import (possbily by using a relevant generator yourself).

Adding a new provider through the REST API¶

Apart from using the UI, it’s also possible to create a provider through the REST API, by POSTing to the http://localhost:6543/providers endpoint to have the server assign an id or by PUTTing to the http://localhost:6543/providers/<id> endpoint to assign your own id, using the following payload:

{
    "id":"CHEESE",
    "conceptscheme_uri":"https://id.python.org/cheese",
    "uri_pattern":"https://id.python.org/cheese/%s",
    "subject":[],
    "id_generation_strategy":"NUMERIC",
    "expand_strategy":"recurse",
    "default_language":"en",
    "force_display_language":""
}

If all goes well, you’ll be greeted with a 201 Created status and your new provider and conceptscheme will be available through the UI.

More information about the Atramhasis API can be found at the http://localhost:6543/api_doc endpoint of your Atramhasis instance or at https://thesaurus.onroerenderfgoed.be/api_doc.

Creating concepts and collections¶

You can now start creating concepts and collections by going to the admin interface at http://localhost:6543/admin.

Hiding a vocabulary¶

Atramhasis allows you to hide a vocabulary. This means the vocabulary is still there as far as services are concerned and you can still edit it. But it will not be visible in the public html user interface. You might want to use it for small and rather technical vocabularies you need but don’t want to draw attention to. The only thing you need to do, is tagging this provider with a subject. By adding the hidden subject to the provider, we let Atramhasis know that this vocabulary should not be present among your regular vocabularies.

Now the STUFF thesaurus will not show up in the public web interface, but REST calls to this conceptscheme will function as normal and you will be able to maintain it from the admin interface.

Force a display language for a vocabulary¶

Under normal circumstances, Atramhasis tries to provide the most appropriate label for a certain concept or collection, based on some default configuration and the preferences of the end-user. Every provider can be marked as having a certain default language (English if not set), but Atramhasis also tries to read what the user wants. It does this through the user’s browser’s locale. This information can be read from the browser’s HTTP headers or cookies. Generally, Atramhasis just knows in what language a user is browsing the site and tries to return labels appropriate for that language. So, the same thesaurus visited from the US will return English labels, while it will return Dutch when visited from Gent (Belgium).

You might have a vocabulary with a strongly preferential relation to a certain language. We ran into this situation with a vocabulary of species: names for plants and trees commonly found in Flanders. Some of them have one or more local, Dutch, names. Most or all of them have an official name in Latin. The normal language handling mechanism created a weird situation. It led to a tree of names that was mostly in Latin, with the odd Dutch word thrown in for good measure. This was not as desired by our users. To that end, a special mechanism was created to force rendering labels of concepts and collections in a certain language, no matter what the end-user’s browser is requesting.

To set this, set the force_display_language attribute of your provider, either through the UI or the REST interface (creating_conceptschemes). Set it to a language supported by the provider (there’s little sense to setting it to a language that isn’t present in the vocabulary). Now Atramhasis will try serving concepts from this provider with this language. All labels will still be shown, but the page title or current label will be set to the selected language as much as possible. The normal language determination mechanisms will keep on working, so if the concept has no label in the requested language, Atramhasis will fall back on other labels present.

Beware that this will only affect the Atramhasis UI, not the Atramhasis REST services. We looked into some solutions for our problem that would have also changed the underlying service, but decided against that because it would have prevented you from making your own choices when interacting with Atramhasis. If you want to render the tree of concepts using a preferred language different from what a browser would advocate for, you can pass the language parameter in a url, eg. http://my.thesaurus.org/conceptschemes/STUFF/tree?language=la.

Internationalisation¶

When you create a new empty project with the atramhasis scaffold, you get an English only version. The standard version of Atramhasis has been translated in Dutch and French. If you desire, you can activate these by editing your project’s development.ini

# Edit and uncomment to activate nl and fr language support or other languages
# you have added yourself.
available_languages = en nl fr

Available languages should be a space separated list of IANA language codes. If you add new languages, please consider contributing them back to the project.

Appearance¶

By implementing a few simple techniques from the Pyramid web framework, it’s very easy to customise the look and feel of the public user interface. The default implementation is a very neutral implementation based on the basic elements in the Foundation framework. Customising and overriding this style is possible if you have a bit of knowledge about HTML and CSS.

You can also override the HTML templates that Atramhasis uses without needing to alter the originals so that future updates to the system will not override your modifications.

Overriding templates¶

One very easy technique to use, is Pyramid’s override assets mechanism. This allows you to override a core Atramhasis template with your own template. Suppose we want to change the text on the Atramhasis homepage to welcome visitors to your instances. This text can be found in atramhasis/templates/welcome.jinja2.

Assuming that you created your project as my_thesaurus, we can now create our own template in my_thesaurus/templates/my_welcome.jinja2. Please consult the Jinja2 documentation if you need help with this.

Once you’ve created your template file, you just need to tell your project to override the default welcome.jinja2 with your version. To do this you need to configure the Pyramid config object found in my_thesaurus/__init__.py.

config.override_asset(
    to_override='atramhasis:templates/welcome.jinja2',
    override_with='my_thesaurus:templates/my_welcome.jinja2'
)

Note

Normally, to see the effect of the changes you made, you would need to restart your webserver. When developing, you can make use of the pserve command’s auto-reload feature. To do this, start your server like this:

$ pserve --reload development.ini

Changing the focal conceptschemes¶

An Atramhasis instance should contain one or more conceptschemes. Four of your conceptschemes can be picked to receive a little more attention and focus than the other ones. These conceptschemes will appear on the homepagina with a list of recently visited concepts in those schemes.

Selecting which conceptschemes receive this focus is done in your development.ini file.

layout.focus_conceptschemes =
  STUFF

This should be a space or newline delimited list, limited to 4 entries.

Changing the CSS¶

Out of the box, Atramhasis, comes with the Zurb Foundation framework. We have created a custom style for this framework, but as always you are free to modify this style. Your custom instance contains a few extension points that make it easy to override and change style elements without having to rewrite to much css. All style related files can be found in the my_thesaurus/static folder. This project’s CSS is being maintained and generated by Compass. You will find a scss folder that contains three files that can be used for easy customisations: _my_thesaurus-settings.scss, _my_thesaurus.scss and _my_thesaurus-admin.scss. The first file is a settings file that allows you to override a lot of variables that are used in generating the css. Suppose you want to override the default row width and the default font. You would change _my_thesarus-settings.scss to the following:

// Custom SASS code for my_thesaurus

$row-width: rem-calc(1140);
$body-font-family: "museo-sans", "Open Sans", "Helvetica", Helvetica, Arial, sans-serif;

To have you changes take effect, you need to recompile the scss files and restart your webserver.

$ compass compile
    write css/app-admin.css
    write css/app.css

The other two files, _my_thesaurus.scss and _my_thesaurus-admin.scss are the final scss files loaded before compiling them and can be used to overwrite things in the public or admin interface.

Security¶

We assume that every deployment of Atramhasis has different needs when it comes to security. Some instances will run on a simple laptop for testing and evaluation purposes, others might need a simple standalone database of users and certain deployments might need to integrate with enterprise authentication systems like LDAP, Active Directory, Single Sign On, …

Atramhasis provides authorisation hooks for security. To edit, add or delete a concept or collection, a user is required to have the ‘editor’ pemission. Unless no authorisation policy has been configured.

To get started, consult the sections of the Pyramid documentation on security.

Prior to version 0.6.3, Atramhasis contained a demo scaffold that had a custom security implementation using Mozilla Persona. Since this service has been discontinued, the security configuration was removed as well. But you can still check out the old code in our Github repository to see how it works.

Sitemap¶

Since Atramhasis 0.7.0 it’s possible to generate a sitemap. It consists of a set of files (one per conceptscheme and an index file) you can submit to a search engine. It will help it index your thesaurus as efficiently as possible.

You can generate the sitemap using the following commands:

# remove any existing sitemaps
$ rm my_thesaurus/static/_sitemaps/*
$ sitemap_generator development.ini

The sitemap index xml will be visible at the root of your webserver, eg. http://localhost:6543/sitemap_index.xml. Depending on how often you edit conceptschemes, concepts or collections it’s a good idea to make this into a cron job. When recreating the sitemap it is best practice to remove existing files from the static/_sitemap directory. If the directory is not empty the script will overwrite existing sitemaps, but unused sitemaps will be retained. Unless the –no-input flag is used, the script wil ask the user to press [enter] before overwriting existing files. The sitemap index will always contain links to all the files (used and unused).

Since a sitemap needs to contain abolute URL’s, the script needs to know where the application is being hosted. This can be controlled with a setting atramhasis.url in the application’s ini file. Set this to the root of your webapplication, eg. http://my.thesaurus.org (no trailing slash needed).

Foreign Keys¶

Atramhasis will often function as a central part of a SOA in an organisation. Concept and maybe Collection objects will be used by other applications. One of the riskier aspects of this is that someone might delete a concept in a certain scheme that is still being used by another application. Even worse, the user approving the delete might not even have a clue that the concept is being used by some external application. While in the decentralised world that is the world wide web, we can never be sure that nobody is using our concept any more, we can take some steps to at least control what happens within other applications that are within our control.

Of course, within the framework that is Atramhasis it’s very difficult to know how or where your own resources might be and how they might be using concepts from Atramhasis. We have therefor provided the necessary hooks for you that can help you deal with the sort of situation. But the actual implementation is left up to you.

We have added a decorator protected_operation(). When you add this decorator to a view, this view will emit a ProtectedResourceEvent. By default we have added this decorator the delete_concept() view.

In you own code, you can subscribe to this ProtectedResourceEvent through the usual pyramid.events.subscriber(). In this event handler you are then free to implement whatever check you need to do. If you find that the resource in question is being used somewhere and this operation should thus not be allowed to proceed, you simply need to raise a atramhasis.protected_resources.ProtectedResourceException. Into this exception you can also pass a list of URI that might provide the user with some feedback as to where this concept might be used.

For example, a sample event handler that would make it impossible to delete concepts with a URI of less than 5 characters:

from pyramid.events import subscriber
from atramhasis.protected_resources import ProtectedResourceEvent

@subscriber(ProtectedResourceEvent)
def never_delete_a_short_uri(event):
    if len(event.uri) < 5:
        raise ProtectedResourceException(
            'resource {0} has a URI shorter than 5 characters, preventing this operation'.format(event.uri),
            []
        )

Adding Analytics¶

Out of the box, it’s very easy to add any analytics integration to Atramhasis. All you need to do is add your tracking snippet to development.ini.

Example code to add Google Analytics tracking:

# Enter your tracking snippet
tracking_snippet = <!-- Google Analytics -->
    <script type="text/javascript">
    (function(i,s,o,g,r,a,m){i["GoogleAnalyticsObject"]=r;i[r]=i[r]||function(){
    (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
    m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
    })(window,document,"script","//www.google-analytics.com/analytics.js","ga");

    ga("create", "some_key", "auto");
    ga("set", "forceSSL", true);
    ga("send", "pageview");
    </script>
    <!-- End Google Analytics -->

Another example to add Plausible Analytics:

# Enter your tracking snippet
tracking_snippet = <script defer data-domain="your.domain.com"
    src="https://your.plausibleinstance.com/js/script.js">
    </script>

Adding external providers¶

Within your Atramhasis instance you can make use of external providers. These are other systems serving up thesauri that you can interact with. Within the admin interface you can create links to these thesauri as SKOS matches. This way you can state that a concept within your thesauri is the same as or similar to a concept in the external thesaurus. And, more interestingly, you can also import concepts from such a thesaurus into your own vocabulary. Importing a concept like this will automatically create a SKOS match for you. Once a match is in place, you can also update your local concept with information from the external concept by performing a merge.

To enable all this power, you need to configure an external provider in your application. Continuing with our example project, we need to go and edit a my_thesaurus/skos/__init__.py. In this file you need to register other instances of skosprovider.providers.VocabularyProvider. Currently providers have already been written for Getty Vocabularies, English Heritage vocabularies and Flanders Heritage Vocabularies. Depending on the system you’re trying to interact with, writing a new provider is fairly simple. For this example, we’ll assume that you want to integrate the wealth of information that the Art and Architecture Thesaurus (AAT) vocabulary offers you.

The AATProvider for this (and other Getty vocabularies) is available as skosprovider_getty and is installed by default in an Atramhasis instance. All you need to do is configure it. First, we need to import the provider. Place this code at the top of my_thesaurus/skos/__init__.py.

from skosprovider_getty.providers import AATProvider

Once this is done, we need to instantiate the provider within the create_registry function and register it with the skosprovider.registry.Registry. By adding the external subject to the provider, we let Atramhasis know that this is not a regular, internal provider that can be stored in our database, but a special external one that can only be used for making matches. As such, it will not be present and visible to the public among your regular vocabularies.

AAT = AATProvider(
    {'id': 'AAT', 'subject': ['external']},
)
registry.register_provider(AAT)

That’s all. You can do the same with the TGNProvider for the Thesaurus of Geographic Names (TGN) or any of the providers for heritagedata.org that can be found in skosprovider_heritagedata.

In the end your my_thesaurus/skos/__init__.py should look somewhat like this:

# -*- coding: utf-8 -*-

import logging
log = logging.getLogger(__name__)

from skosprovider.registry import Registry
from skosprovider_getty.providers import AATProvider
from atramhasis.skos import register_providers_from_db


def create_registry(request):
    # create the SKOS registry
    registry = Registry(instance_scope='threaded_thread')

    AAT = AATProvider(
        {
            'id': 'AAT',
            'subject': ['external']
        }
    )

    registry.register_provider(AAT)

    register_providers_from_db(registry, request.db)

    return registry

Now you’ll be able to import from the AAT to your heart’s delight. For an extended example that adds even more providers, you could have a look at the demo scaffold that comes with Atramhasis.

Import a controlled vocabulary¶

Atramhasis includes a script atramhasis/scripts/import_file.py which helps you import an existing vocabulary from a file. It supports a few different file types, but not every file type supports the full Atramhasis datamodel.

The supported file types:

RDF using RDFProvider. This provider supports the full datamodel. Since the heavy lifting is done by RDFlib, most of the dialects supported by RDFlib should work. The full list can be found in rdflib.util.SUFFIX_FORMAT_MAP. Formats like rdf/xml and turtle should work.
CSV (.csv) using SimpleCsvProvider. The provider only supports importing and id, a prefLabel, a note and a source. It will work well when importing a simple flat list, but not for complex hierarchies.
JSON (.json) using DictionaryProvider. This provider supports the full datamodel.

Some things to take into account:

When importing from an RDF vocabulary, the id will be read from a dc or dcterms identifier property if present. If you have local identifiers, please make sure to add them using this property. If not, your concept and collections will use their URI’s as id.
When importing from RDF, the import file could possibly contain more than one conceptscheme. Please ensure only one conceptscheme is present or no conceptschemes are presents and specify the URI and label on the command line.
When importing from CSV or JSON, the data file only contains the concepts and collections in the scheme, but not the conceptscheme itself. In this case, please specify the URI and label of the conceptscheme on the command line.

The script can be called through the commandline in the project virtual environment. Call it with the help argument to see the possible arguments.

$ workon my_thesarus
$ import_file --help

usage: import_file.py [-h] [--to conn_string] [--conceptscheme-label CS_LABEL]
                      [--conceptscheme-uri CS_URI]
                      [--create-provider | --no-create-provider]
                      [--provider-id PROVIDER_ID]
                      [--id-generation-strategy {numeric,guid,manual}]
                      input_file uri_pattern

Import file to a database

positional arguments:
  input_file            local path to the input file
  uri_pattern           URI pattern input for the URIGenerator

options:
  -h, --help            show this help message and exit
  --to conn_string      Connection string of the output database
  --conceptscheme-label CS_LABEL
                        Label of the conceptscheme
  --conceptscheme-uri CS_URI
                        URI of the conceptscheme
  --create-provider, --no-create-provider
                        An optional parameter if given a provider is created.
                        Use --no-create-provider to not create a provider
                        (default: True)
  --provider-id PROVIDER_ID
                        An optional string (eg. ERFGOEDTYPES) to be assigned
                        to the provider id. If not specified, assign the
                        conceptscheme id to the provider id
  --id-generation-strategy {numeric,guid,manual}
                        URI pattern input for the URIGenerator

example: import_file.py atramhasis/scripts/my_file urn:x-skosprovider:trees:%s
--to sqlite:///atramhasis.sqlite --conceptscheme-label Labels --conceptscheme-
uri urn:x-skosprovider:trees --create-provider --provider-id ERFGOEDTYPES
--id-generation-strategy numeric

The input_file is a positional required argument and details where the file you want to import is located, for example my_thesaurus/data/trees.json. It is relative to your current location.

The uri_pattern is a positional required argument that details the URI pattern that will be used by the provider to create concept and collection URI’s.

The to argument contains the connection string of output database. Only PostGreSQL and SQLite are supported. The structure is either postgresql://username:password@host:port/db_name or either sqlite:///path/db_name.sqlite. The default value is sqlite:///atramhasis.sqlite.

The data is loaded in a ConceptScheme. With a RDFProvider the conceptscheme can be present in the RDF file. The other providers can specify it on the command line through the conceptscheme_label argument. If no conceptscheme_label is present, the default label is the name of the file.

Once the data is loaded in the database, your conceptscheme and provider will be ready, unless you chose the –no-create-provider option. This is rarely what you want and is only useful if you are doing some manual interventions in your database.

For example, to insert this file:

[{"broader": [],
  "id": 1,
  "labels": [{"label": "The Larch",
               "language": "en",
               "type": "prefLabel"},
              {"label": "De Lariks",
               "language": "nl",
               "type": "prefLabel"}],
  "matches": {"broad": [],
               "close": [],
               "exact": [],
               "narrow": [],
               "related": []},
  "member_of": [3],
  "narrower": [],
  "notes": [{"language": "en",
              "note": "A type of tree.",
              "type": "definition"}],
  "related": [],
  "subordinate_arrays": [],
  "type": "concept",
  "uri": "http://id.trees.org/1"},
 {"broader": [],
  "id": 2,
  "labels": [{"label": "The Chestnut",
               "language": "en",
               "type": "prefLabel"},
              {"label": "De Paardekastanje",
               "language": "nl",
               "type": "altLabel"},
              {"label": "la châtaigne",
               "language": "fr",
               "type": "altLabel"}],
  "matches": {"broad": [],
               "close": [],
               "exact": [],
               "narrow": [],
               "related": []},
  "member_of": [3],
  "narrower": [],
  "notes": [{"language": "en",
              "note": "A different type of tree.",
              "type": "definition"}],
  "related": [],
  "subordinate_arrays": [],
  "type": "concept",
  "uri": "http://id.trees.org/2"},
 {"id": 3,
  "labels": [{"label": "Bomen per soort",
               "language": "nl",
               "type": "prefLabel"},
              {"label": "Trees by species",
               "language": "en",
               "type": "prefLabel"}],
  "member_of": [],
  "members": [1, 2],
  "notes": [],
  "superordinates": [],
  "type": "collection",
  "uri": "http://id.trees.org/3"}]

We run the following command:

$ workon my_thesarus
$ import_file my_thesaurus/data/trees.json <uri-pattern> --to sqlite:///my_thesaurus.sqlite --conceptscheme_label Trees

This will return output similar to this:

sqlalchemy.engine.base.Engine SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
sqlalchemy.engine.base.Engine ()
sqlalchemy.engine.base.Engine SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
sqlalchemy.engine.base.Engine ()
sqlalchemy.engine.base.Engine BEGIN (implicit)
sqlalchemy.engine.base.Engine INSERT INTO note (note, notetype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('A type of tree.', 'definition', 'en')
sqlalchemy.engine.base.Engine INSERT INTO note (note, notetype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('A different type of tree.', 'definition', 'en')
sqlalchemy.engine.base.Engine INSERT INTO conceptscheme (uri) VALUES (?)
sqlalchemy.engine.base.Engine (None,)
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('Trees', 'prefLabel', 'nl')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('The Larch', 'prefLabel', 'en')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('De Lariks', 'prefLabel', 'nl')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('The Chestnut', 'prefLabel', 'en')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('De Paardekastanje', 'altLabel', 'nl')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('la châtaigne', 'altLabel', 'fr')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('Bomen per soort', 'prefLabel', 'nl')
sqlalchemy.engine.base.Engine INSERT INTO label (label, labeltype_id, language_id) VALUES (?, ?, ?)
sqlalchemy.engine.base.Engine ('Trees by species', 'prefLabel', 'en')
sqlalchemy.engine.base.Engine INSERT INTO conceptscheme_label (conceptscheme_id, label_id) VALUES (?, ?)
sqlalchemy.engine.base.Engine (11, 3548)
sqlalchemy.engine.base.Engine INSERT INTO concept (type, concept_id, uri, conceptscheme_id) VALUES (?, ?, ?, ?)
sqlalchemy.engine.base.Engine ('concept', 1, 'http://id.trees.org/1', 11)
sqlalchemy.engine.base.Engine INSERT INTO concept (type, concept_id, uri, conceptscheme_id) VALUES (?, ?, ?, ?)
sqlalchemy.engine.base.Engine ('concept', 2, 'http://id.trees.org/2', 11)
sqlalchemy.engine.base.Engine INSERT INTO concept (type, concept_id, uri, conceptscheme_id) VALUES (?, ?, ?, ?)
sqlalchemy.engine.base.Engine ('collection', 3, 'http://id.trees.org/3', 11)
sqlalchemy.engine.base.Engine INSERT INTO concept_label (concept_id, label_id) VALUES (?, ?)
sqlalchemy.engine.base.Engine ((2558, 3551), (2558, 3552), (2558, 3553), (2557, 3549), (2557, 3550), (2559, 3554), (2559, 3555))
sqlalchemy.engine.base.Engine INSERT INTO concept_note (concept_id, note_id) VALUES (?, ?)
sqlalchemy.engine.base.Engine ((2558, 3605), (2557, 3604))
sqlalchemy.engine.base.Engine SELECT concept.id AS concept_id_1, concept.type AS concept_type, concept.concept_id AS concept_concept_id, concept.uri AS concept_uri, concept.conceptscheme_id AS concept_conceptscheme_id
FROM concept
WHERE concept.conceptscheme_id = ? AND concept.concept_id = ? AND concept.type IN (?)
sqlalchemy.engine.base.Engine (11, 1, 'concept')
sqlalchemy.engine.base.Engine SELECT concept.id AS concept_id_1, concept.type AS concept_type, concept.concept_id AS concept_concept_id, concept.uri AS concept_uri, concept.conceptscheme_id AS concept_conceptscheme_id
FROM concept
WHERE concept.conceptscheme_id = ? AND concept.concept_id = ? AND concept.type IN (?)
sqlalchemy.engine.base.Engine (11, 2, 'concept')
sqlalchemy.engine.base.Engine SELECT concept.id AS concept_id_1, concept.type AS concept_type, concept.concept_id AS concept_concept_id, concept.uri AS concept_uri, concept.conceptscheme_id AS concept_conceptscheme_id
FROM concept
WHERE concept.conceptscheme_id = ? AND concept.concept_id = ? AND concept.type IN (?)
sqlalchemy.engine.base.Engine (11, 3, 'collection')
sqlalchemy.engine.base.Engine SELECT concept.id AS concept_id_1, concept.type AS concept_type, concept.concept_id AS concept_concept_id, concept.uri AS concept_uri, concept.conceptscheme_id AS concept_conceptscheme_id
FROM concept
WHERE concept.conceptscheme_id = ? AND concept.concept_id = ?
sqlalchemy.engine.base.Engine (11, 1)
sqlalchemy.engine.base.Engine SELECT concept.id AS concept_id_1, concept.type AS concept_type, concept.concept_id AS concept_concept_id, concept.uri AS concept_uri, concept.conceptscheme_id AS concept_conceptscheme_id
FROM concept, collection_concept
WHERE ? = collection_concept.collection_id AND concept.id = collection_concept.concept_id
sqlalchemy.engine.base.Engine (2559,)
sqlalchemy.engine.base.Engine INSERT INTO collection_concept (collection_id, concept_id) VALUES (?, ?)
sqlalchemy.engine.base.Engine (2559, 2557)
sqlalchemy.engine.base.Engine SELECT concept.id AS concept_id_1, concept.type AS concept_type, concept.concept_id AS concept_concept_id, concept.uri AS concept_uri, concept.conceptscheme_id AS concept_conceptscheme_id
FROM concept
WHERE concept.conceptscheme_id = ? AND concept.concept_id = ?
sqlalchemy.engine.base.Engine (11, 2)
sqlalchemy.engine.base.Engine INSERT INTO collection_concept (collection_id, concept_id) VALUES (?, ?)
sqlalchemy.engine.base.Engine (2559, 2558)
sqlalchemy.engine.base.Engine COMMIT
sqlalchemy.engine.base.Engine BEGIN (implicit)
sqlalchemy.engine.base.Engine SELECT label.id AS label_id, label.label AS label_label, label.labeltype_id AS label_labeltype_id, label.language_id AS label_language_id
FROM label JOIN conceptscheme_label ON label.id = conceptscheme_label.label_id
WHERE label.label = ?
 LIMIT ? OFFSET ?
sqlalchemy.engine.base.Engine ('Trees', 1, 0)
sqlalchemy.engine.base.Engine SELECT conceptscheme.id AS conceptscheme_id, conceptscheme.uri AS conceptscheme_uri
FROM conceptscheme, conceptscheme_label
WHERE ? = conceptscheme_label.label_id AND conceptscheme.id = conceptscheme_label.conceptscheme_id
sqlalchemy.engine.base.Engine (3548,)


*** The import of the my_thesaurus/data/trees.json file with conceptscheme label 'Trees' to sqlite:///my_thesaurus.sqlite was successfull. ***

Now your thesaurus has been successfully imported and is ready to be browsed, expanded and edited.

SessionFactory¶

You can change the default session factory in the __init__.py file.

# set default session factory
from pyramid.session import SignedCookieSessionFactory
atramhasis_session_factory = SignedCookieSessionFactory(settings['atramhasis.session_factory.secret'])
config.set_session_factory(atramhasis_session_factory)

Updating an older installation of Atramhasis¶

If you are running an older installation of Atramhasis, it’s important you reconfigure how providers are created. While they were created in pre-2.0.0 version by writing a little bit of code, since 2.0.0 they are created through the UI or the REST service and stored in the database. Run the following command:

$ workon my_thesaurus
$ migrate_sqlalchemy_providers --setings_file development.ini

After running this command, all providers will be present in the DB and you can safely delete some code in the my_thesaurus/skos/__init__.py. You should edit you create_registry function. Where you would previously have instantiated all SQLAlchemyProviders in this function, you need to remove them and replace them with a call to create_providers_from_db(). Any external providers you registered, such as AAT providers, should still be configured through code in this file. Your final code should looks somewhat like this:

# -*- coding: utf-8 -*-

import logging
log = logging.getLogger(__name__)

from skosprovider.registry import Registry
from skosprovider_getty.providers import AATProvider
from atramhasis.skos import register_providers_from_db


def create_registry(request):
    # create the SKOS registry
    registry = Registry(instance_scope='threaded_thread')

    AAT = AATProvider(
        {
            'id': 'AAT',
            'subject': ['external']
        }
    )

    registry.register_provider(AAT)

    # Load all configured providers from the db
    register_providers_from_db(registry, request.db)

    return registry

Customisation¶

Creating your own project¶

Database¶

Database initialisation¶

Custom alembic revisions¶

Running a local server¶

Creating conceptschemes¶

Adding a new provider through the UI¶

Adding a new provider through the REST API¶

Creating concepts and collections¶

Hiding a vocabulary¶

Force a display language for a vocabulary¶

Internationalisation¶

Appearance¶

Overriding templates¶

Changing the focal conceptschemes¶

Changing the CSS¶

Security¶

Sitemap¶

Foreign Keys¶

Adding Analytics¶

Adding external providers¶

Import a controlled vocabulary¶

SessionFactory¶

Updating an older installation of Atramhasis¶

Table of Contents

Previous topic

Next topic

This Page