Cloud Endpoints (2.0.2)

The documentation on this page corresponds to Version 2.0.2 rev 218 (December 2017) of the tools suite. See Older Versions for documentation on earlier releases of this suite.

There are currently two options for Cloud Endpoints to communicate with ODK 2 tools.

See below for more information about the cloud endpoints.

 


ODK Sync Endpoint

 

ODK Sync Endpoint is a Docker container that implements the ODK 2.0 sync protocol.

Authentication

ODK Sync Endpoint does not store user information in its own database, instead it integrates with a LDAP directory or an Active Directory. That directory is then used to authenticate users and obtain user roles.

NOTE: As a consequence of the integration, Basic Authentication is the only supported authentication method.

ODK Sync Endpoint prerequisites

You must have Docker 17.06.1 or newer, and be running in Swarm Mode.
Follow these links for detailed instructions on installing Docker and enabling Swarm Mode.

ODK Sync Endpoint Setup

ODK Sync Endpoint requires a database and a LDAP directory, you could follow the instructions and deploy all 3 components together or supply your own database and/or LDAP directory.

NOTE: All of the following command should be ran on your server

  1. git clone https://github.com/opendatakit/sync-endpoint-default-setup
  2. docker build --pull -t odk/sync_endpoint https://github.com/opendatakit/sync-endpoint-containers.git
  3. docker build --pull -t odk/sync-web-ui https://github.com/opendatakit/sync-endpoint-web-ui.git
  4. In the cloned repository,
    docker build --pull -t odk/db-bootstrap db-bootstrap
  5. In the cloned repository,
    docker build --pull -t odk/openldap openldap
  6. In the cloned repository,
    docker build --pull -t odk/phpldapadmin phpldapadmin
  7. Enter your hostname in the
    security.server.hostname

    field in

    security.properties
  8. If you're not using the standard ports, i.e. 80 for HTTP and 443 for HTTPS, enter the ports you're using in the
    security.server.port

    and

    security.server.securePort

    fields in

    security.properties

    Then edit the

    ports

    section under the

    sync

    section in

    docker-compose.yml

    to be

    "YOUR_PORT:8080"

    It is important that the second parts stays as 8080.

  9. If you're using your own LDAP directory or database, continue with the instructions below.
  10. In the cloned repository,
    docker stack deploy -c docker-compose.yml syncldap
  11. The server takes about 30s to start, then it will be running at http://127.0.0.1.
  12. See the LDAP section for instructions on configuring users and groups
Custom database
  1. If you haven't followed the common instructions above, start with those.
  2. Remove the
    db

    and

    db-bootstrap

    sections in

    docker-compose.yml
  3. Modify
    jdbc.properties

    to match your database. Supported database systems are PostgreSQL, MySQL and Microsoft SQL Server.
    Sample config for each type of database can be found here

  4. Modify
    sync.env

    to match your database

  5. In the cloned repository,
    docker stack deploy -c docker-compose.yml syncldap
  6. The server takes about 30s to start, then it will be running at http://127.0.0.1.
Custom LDAP directory
  1. If you haven't followed the common instructions above, start with those.
  2. OPTIONAL: If your LDAP directory uses a certificate that was signed by a self-signed CA,
    1. Make the public key of the CA available to ODK Sync Endpoint with this command.
      docker config create org.opendatakit.sync.ldapcert PATH_TO_CERT
    2. Uncomment the relevant lines in the
      configs

      section in

      docker-compose.yml

      and the

      configs

      section under the

      sync

      section in

      docker-compose.yml
  3. Remove the
    ldap-service

    and

    phpldapadmin

    sections in

    docker-compose.yml
  4. Modify the relevant sections in
    security.properties

    to match your LDAP directory. Further instructions are in the file.
    NOTE: the default configuration does not use ldaps or StartTLS because the LDAP directory communicates with the ODK Sync Endpoint over a secure overlay network. You should use ldaps or StartTLS to communicate with your LDAP directory.

  5. In the cloned repository,
    docker stack deploy -c docker-compose.yml syncldap
  6. The server takes about 30s to start, then it will be running at http://127.0.0.1.

Stopping ODK Sync Endpoint

  1. docker stack rm syncldap
  2. OPTIONAL: If you want to remove the volumes as well,
    Linux/MacOS:
    docker volume rm $(docker volume ls -f "label=com.docker.stack.namespace=syncldap" -q)

    Windows:

    docker volume rm (docker volume ls -f "label=com.docker.stack.namespace=syncldap" -q)

LDAP

The default admin account is cn=admin,dc=example,dc=org. The default password is admin, it can be changed with the LDAP_ADMIN_PASSWORD environment variable in ldap.env.

The default readonly account is cn=readonly,dc=example,dc=org. The defualt password is readonly, it can be changed with the LDAP_READONLY_USER_PASSWORD environment variable in ldap.env. This account is used by the Sync Endpoint to retrieve user information.

Ther LDAP directory that you deployed with the instructions above is an OpenLDAP server. In addition to the directory, a phpLDAPadmin server is also deployed to help you configure the directory.
If you'd perfer to use the OpenLDAP command line utilities, they're installed in the OpenLDAP container. These tools are accessible with this command,
Linux/MacOS:

docker exec $(docker ps -f "label=com.docker.swarm.service.name=syncldap_ldap-service" --format '{{.ID}}') LDAPTOOL ARGS

Windows:

docker exec (docker ps -f "label=com.docker.swarm.service.name=syncldap_ldap-service" --format '{{.ID}}') LDAPTOOL ARGS

NOTE: the phpLDAPadmin server listens on port 40000, it is important that you do not expose this port to the internet.

 

The following guides assume that you're using phpLDAPadmin.

Creating users
  1. Click
    login

    on the right and login as admin

  2. Expand the tree view on the right until you see
    ou=people
  3. Click on
    ou=people

    and choose

    Create a child entry
  4. Choose the
    Generic: User Account

    template

  5. Fill out the form and click create object
  6. Refer to the section below on assigning this user to groups
Creating groups
  1. Click
    login

    on the right and login as admin

  2. Expand the tree view on the right until you see
    ou=groups
  3. Click on
    ou=default_prefix

    and choose

    Create a child entry
  4. Choose the
    Generic: Posix Group

    template

  5. Fill out the form and click create object
    Note: the group name must start with the group prefix, in this case the group prefix is
    default_prefix

    , e.g.

    default_prefix my-new-group
  6. Refer to the section below on assigning users to this group
Assigning users to groups
  1. Click
    login

    on the right and login as admin

  2. Expand the tree view on the right until you see
    ou=default_prefix

    , then expand

    ou=default_prefix
  3. This list is all the groups under
    ou=default_prefix
  4. Click on the group that you want to assign users to
  5. A few groups are created when the LDAP server is brought up, refer to Data Permission Filters for descriptions of these groups.
  6. If the
    memberUid

    section is not present,

    1. Choose
      Add new attribute
    2. Choose
      memberUid

      from the dropdown, then enter

      uid

      of the user you want to assign

    3. Click update object at the bottom to update
  7. If the
    memberUid

    section is present,

    1. Navigate to the
      memberUid

      section

    2. Click modify group members to manage members

HTTPS

  1. Store your certificate public key in a Docker config with this command,
    docker config create example.com.fullchain.pem PATH_TO_PUBLIC_KEY
  2. Store your certificate private key in a Docker secret with this command,
    docker secret create examepl.com.privkey.pem PATH_TO_PRIVATE_KEY
  3. Modify the configs section and secrets section in docker-compose.yml to include name of the Docker config and Docker secret created above.
  4. Uncomment the relevant lines in the nginx section in docker-compose.yml

 


ODK Aggregate Tables Extension

 

Release Designation

The ODK 2.0 functionality within ODK Aggregate 1.4.15 (July 2017) is designated as Release Candidate software. See Release Designations for the meaning of this designation.

To determine the ODK Aggregate release, log in with Site Admin privileges and go to the Site Admin / Preferences sub-tab.

Overview

The ODK Aggregate Tables Extensions enable the ODK 2.0 tools to share data via bi-directional synchronization with a central ODK Aggregate server.

The ODK 2.0.2 rev 218 sync protocol is compatible with ODK Aggregate v1.4.15. The sync protocol has been augmented to cache the user's permissions on the device and, for super-users or administrators, to cache the full set of users and all of their permissions (so that the super-user and/or administrator can assign rows to particular individuals).

This documents how to set up ODK Aggregate to accept sync and how to perform a sync and a data-table-download using ODK Services v2.0.0 .

 

Server Setup

First you’ll have to install ODK Aggregate v1.4.15 to a server (see the Aggregate page for these instructions).

  1. Install ODK Aggregate v1.4.15 to a server.
  2. Log onto your ODK Aggregate v1.4.15 instance.
  3. Go to the Site Admin / Preferences page.
  4. Check the checkbox for "ODK Tables Synchronization Functionality"
  5. Go to the Site Admin / Permissions page.
  6. Add ODK Aggregate usernames or Gmail or Google Apps account users (do this by typing one or more users' usernames or e-mail addresses into the text area and clicking Add User).
  7. If you have created an ODK Aggregate username, be sure to Change Password on that account to set the initial password for the account.
  8. Grant these users the Synchronize Tables permissions.
  9. Select at least one user to be the administrator and grant them Administer Tables permissions. This user will have the ability to 'Reset App Server' from the Android device and add or remove tables and configuration files on the server. This is the equivalent of the Form Manager permissions in ODK 1.x deployments.
  10. Click 'Save Changes'. These changes will not take effect until you do!

Changing the AppName

The ODK 2.0 tools are designed to support multiple, independent, ODK 2.0 applications running on the Android device. Each of our tools has the ability to run in the context of either a default application name, or a specified application name.

By default, all the ODK 2.0 tools run under the default application name. Application names correspond to the name of the directory under /sdcard/opendatakit where the data files for that application are stored.

When you run ODK Services from within ODK Survey, the ODK Survey tool informs ODK Services to run in the context of the application name under which the ODK Survey tool is running. When ODK Services then interacts with ODK Aggregate, it reports that application name to the server. The server must be configured with exactly the same application name or it will reject the requests from ODK Services. The same applies when launching ODK Services from within ODK Tables.

ODK Aggregate is configured by default to use the 'default' application name. To change the name, go to the Site Admin / Preferences screen and click the 'Change ODK 2.0 App Name' button, and enter a new application name. For example, the https://opendatakit-surveydemo.appspot.com server is configured with 'survey' as its application name.

Using Device Synchronization

For more information on syncing, click here.

Distribution: 
2.x Release Version: