Search

Domino Upgrade

VersionSupport end
5.0
6.0
6.5
7.0
8.0
8.5
Upgrade to 9.x now!
(see the full Lotus lifcyle) To make your upgrade a success use the Upgrade Cheat Sheet.
Contemplating to replace Notes? You have to read this! (also available on Slideshare)

Languages

Other languages on request.

Twitter

Useful Tools

Get Firefox
Use OpenDNS
The support for Windows XP has come to an end . Time to consider an alternative to move on.

About Me

I am the "IBM Collaboration & Productivity Advisor" for IBM Asia Pacific. I'm based in Singapore.
Reach out to me via:
Follow notessensei on Twitter
(posts)
Skype
Sametime
IBM
Facebook
LinkedIn
XING
Amazon Store
Amazon Kindle
NotesSensei's Spreadshirt shop
profile for stwissel on Stack Exchange, a network of free, community-driven Q&A sites

02/04/2016

Annotations to supercharge your vert.x development

Category
ProjectCastle is well under way. Part of it, the part talking to Domino, is written in Java8 and vert.x. With some prior experience in node.js development vert.x will look familiar: base on event loop and callbacks, you develop in a very similar way. The big differences: vert.x runs on the JVM8, it is by nature of the JVM multi-threaded, features an event bus and is polyglot - you can develop in a mix of languages: Java, JavaScript, Jython, Groovy etc.
This post reflects some of the approaches I found useful developing with vert.x in Java. There are 3 components which are core to vert.x development:
  • Verticle

    A unit of compute running with an event loop. Usually you start one Verticle (optional with multiple instances) as your application, but you might want/need to start additional ones for longer running tasks. A special version is the worker verticle, that runs from a thread pool to allow execution of blocking operations
  • EventBus

    The different components of your application message each other via the EventBus. Data send over the EventBus can be a String, a JsonObject or a buffer. You also can send any arbitrary Java class as message once you have defined a codec for it
  • Route

    Like in node.js a vert.x web application can register routes and their handlers to react on web input under various conditions. Routes can be defined using URLs, HTTP Verbs, Content-Types ( for POST/PUT/PATCH operations)
Ideally when defining a route and a handler, a verticle or a potential message for the EventBus, all necessary code stays contained in the respective source code file. The challenge here is to register the components when the application starts. Your main Verticle doesn't know what components are in your application and manually maintain a loader code is a pain to keep in sync (besides leading to merge conflicts when working in a team).
Java annotations to the rescue! If you are new to annotations, go and check out this tutorial to get up to speed. For my project I defined three of them, with one being able to be applied multiple times.

CastleRequest

A class annotated with CastleRequest registers its handler with the EventBus, so the class can be sent over the EventBus and get encoded/decode appropriately. A special value for the annotation is "self" which indicates, that the class itself implements the MessageCodec interface
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface CastleRequest {
  // We use value to ease the syntax
  // to @CastleRequest(NameOfCodec)
  // Special value: self = class implements the MessageCodec interface
  String value();
}

CastleRoute

This annotation can be assigned multiple times, so 2 annotation interfaces are needed
@Documented
@Repeatable(CastleRoutes.class)
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface CastleRoute {
  String route();
  String description();
  String mimetype() default "any";
  String method() default "any";
}

and the repeatability annotation (new with Java8):
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface CastleRoutes {
  CastleRoute[] value();
}

CastleVerticle

Classes marked with this annotation are loaded as verticles. They can implement listeners to the whole spectrum of vert.x listening capabilities
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface CastleVerticle {
  String type() default "worker";
  int instances() default 0;
  boolean multithreaded() default false;
}

25/02/2016

Now we are token - Authorization using JSON Web Token in Domino

Category  
After having Vert.x and Domino co-exist, the door opens for a few interesting applications of the new found capabilites. One sticky point in each application landscape is authentication and authorization. This installment is about authorization.
The typical flow:
  1. you access a web resource
  2. provide some identity mechanism (in the simplest case: username and password)
  3. in exchange get some prove of identity
  4. that allows you to access protected resources.
In Basic authentication you have to provide that prove every time in form of an encoded username/password header. Since that limits you to username and password, all other mechanism provide you in return for your valid credentials with a "ticket" (technically a "Bearer Token") that opens access.
I tend to compare this with a movie theater: if you want to enter the room where the movie plays you need a ticket. The guy checking it, only is interested: is it valid for that show. He doesn't care if you paid in cash, with a card, got it as a present or won in a lucky draw. Did you buy it just now, or online or yesterday, he doesn't care. He cares only: is it valid. Same applies to our web servers.
In the IBM world the standard here is an LTPA token that gets delivered as cookie. Now cookies (besides being fattening) come with their own little set of trouble and are kind of frowned upon in contemporary web application development.
The current web API token darling is JSON Web Token (JWT). They are an interesting concept since they sign the data provided. Be clear: they don't encrypt it, so you need to be careful if you want to store sensitive information (and encrypt that first).

Now how to put that into Domino?

The sequence matches the typical flow:
  1. User authenticates with credentials
  2. server creates a JWT
  3. stores JWT and credentials in a map, so when the user comes back with the token, the original credentials can be retrieved
  4. delivers JWT to caller
  5. Caller uses JWT for next calls in the header
It isn't rocket science to get that to work.

18/02/2016

Vert.x and Domino

Category  
A while ago I shared how to use vert.x with a Notes client, which ultimately let me put an Angular face on my inbox and inspired the CrossWorlds project.
I revisited vert.x which is now 3.2.1 and no longer beta. On a Domino Linux server (I don't have Windows) and on a Mac Notes client the JVM is 64 Bit, which makes the configuration easier (no -w32 switch, no download of an additional JVM). The obligatory HelloWorld verticle ran quite nicely with my manually. However it wouldn't run, when the Domino ran using a startup script.
The simple reason: to be able to access the Domino instance the vert.x verticle needs to run with the same user as the Domino server. su into the user doesn't do the trick - and of course you can't login into my server with the id that runs Domino. The solution was to turn to the expert and his outstanding Linux boot script. Using the /etc/sysconfig/rc_domino_config_* file you can simply define the behavior of your Domino startup and shutdown experience. Mine looks like this (I use "domino" as my standard user, not "notes"):

rc_domino_config_domino

LOTUS=/opt/ibm/domino
DOMINO_DATA_PATH=/home/domino/notesdata
DOMINO_SHUTDOWN_TIMEOUT=600
DOMINO_CONFIGURED="yes"
BROADCAST_SHUTDOWN_MESSAGE="yes"
DOMINO_REMOVE_TEMPFILES="yes"
DOMINO_POST_STARTUP_SCRIPT=/home/domino/scripts/launch_vertx
DOMINO_PRE_SHUTDOWN_SCRIPT=/home/domino/scripts/stop_vertx

I have installed vert.x using npm using the full stack. With node.js installed, all you need is sudo npm install vertx3-full. Of course there are more conservative ways to install, vert.x, this may be an exercise left to the reader. I didn't use any of the environment variables exposed by the standard boot script to keep it independent. The script itself is just a few lines:

launch_vertx

#!/bin/sh
# Starts the vert.x tasks that talks to Domino
DOMINO_HOME=/opt/ibm/domino/notes/latest/linux
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export VERTX_HOME=/usr/lib/node_modules/vertx3-full/vertx
export DYLD_LIBRARY_PATH=$DOMINO_HOME
export LD_LIBRARY_PATH=$DOMINO_HOME
export CLASSPATH=.:$DOMINO_HOME/jvm/lib/ext/Notes.jar:$CLASSPATH
export NOTES_ENV=SERVER
vertx start com.ibm.issc.verseu.VerseLauncher -cp /home/domino/scripts/verseu.jar --vertx-id domino

The shutdown script is short and sweet. Since I used an vertx-id, I can use that to shut down the verticle without knowing or caring for its startclass name:

stop_vertx

#!/bin/sh
# Stops the vert.x tasks that talks to Domino
export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export VERTX_HOME=/usr/lib/node_modules/vertx3-full/vertx
vertx stop domino

Next step: write some actual code beyond "Hello World".
As usual YMMV

26/09/2014

Rendering a Notes view as JSON REST service - on your client

Category  
My next goal after getting the basic connection to Notes working is to be able to serve a potential API. Still making friends with the non-blocking approach of vert.x, I'm taking baby steps forward. In this round I want to be able to deliver a view or folder as JSON string. On a Domino server that is easy. You can use ?ReadViewEntries&OutputFormat=JSON. On a Notes client you have to do it yourself.
In round one I will ignore categorized views (that's for the next time), but I already will massage the JSON to be leaner. After all why send it over the wire what you don't need. So I have a little AppConfig.INSTANCE singleton, that delivers a viewConfig object. This object has the list of columns and the inteded labels that I want to be returned.
Since last time some of the libraries have been updated and I'm now running vert.x 3.0.0.Preview1 and the OpenNTF Domino API RC2. I unpacked the OpenNTF release and removed the Jar files and replaced them with Maven dependencies. This step isn't necessary, but I'm expanding my Maven knowledge, so it was good practise. The starter application looks quite simple:
package com.notessensei.vertx.notes;

import io.vertx.core.Handler;
import io.vertx.core.Vertx;
import io.vertx.core.http.HttpServer;
import io.vertx.core.http.HttpServerOptions;
import io.vertx.core.http.HttpServerRequest;
import java.io.IOException;
import org.openntf.domino.thread.DominoExecutor;


public class NotesClient {

    /**
     * @param args
     * @throws IOException
     */
    public static void main(String[] args) throws IOException {
        final NotesClient nc = new NotesClient();
        nc.runUntilKeyPresses("q");
        System.exit(0);
    }

    private static final int     listenport        = 8110;
    private static final int     dominothreadcount = 10;
    private final Vertx          vertx;
    private final HttpServer     hs;
    private final DominoExecutor de;

    public NotesClient() throws IOException {
        this.vertx = Vertx.factory.vertx();
        final HttpServerOptions options = HttpServerOptions.options();
        options.setPort(NotesClient.listenport);
        this.hs = this.vertx.createHttpServer(options);
        this.de = new DominoExecutor(NotesClient.dominothreadcount);
    }

    public void runUntilKeyPresses(String keystring) throws IOException {
        int quit = 0;
        final int quitKey = keystring.charAt(0);

        this.startListening();

        while (quit != quitKey) { // Wait for a keypress
            System.out.print("Notes Client Verticle started, version ");
            System.out.println(AppConfig.INSTANCE.getVersion());
            System.out.print("Started to listen on port ");
            System.out.println(NotesClient.listenport);
            System.out.print("Press ");
            System.out.print(keystring);
            System.out.println("<Enter> to stop the Notes Client Verticle");
            quit = System.in.read();
        }

        this.stopListening();

        System.out.println("\n\nNotes Client Verticle terminated!");
    }

    private void startListening() {
        final Handler<HttpServerRequest> h = new NotesRequestHandler(this.de);
        this.hs.requestHandler(h).listen();
    }

    private void stopListening() {
        this.hs.close();
        this.de.shutdown();
        AppConfig.INSTANCE.save();
    }
}

The Notes request handler, checks what is requested and renders the view into JSON using a "homegrown" JSONBuilder which I designed similar to a SAX writer.
package com.notessensei.vertx.notes;

import java.util.Map;

import io.vertx.core.Handler;
import io.vertx.core.http.HttpServerRequest;
import io.vertx.core.http.HttpServerResponse;

import org.openntf.domino.Database;
import org.openntf.domino.Session;
import org.openntf.domino.View;
import org.openntf.domino.ViewEntry;
import org.openntf.domino.ViewNavigator;
import org.openntf.domino.thread.AbstractDominoRunnable;
import org.openntf.domino.thread.DominoExecutor;
import org.openntf.domino.thread.DominoSessionType;

public class NotesRequestHandler extends AbstractDominoRunnable implements Handler<HttpServerRequest> {

    private static final long           serialVersionUID = 1L;
    private transient HttpServerRequest req;
    private ViewConfig                  viewConfig       = null;
    private final DominoExecutor        de;

    public NotesRequestHandler(DominoExecutor de) {
        this.de = de;
        this.setSessionType(DominoSessionType.NATIVE);
    }

    @Override
    public void run() {
        Session s = this.getSession();
        HttpServerResponse resp = this.req.response();
        this.renderInbox(s, resp);
    }

    public void handle(HttpServerRequest req) {
        HttpServerResponse resp = req.response();

        String path = req.path();

        String[] pathparts = path.split("/");
        // The request must have notes in the URL
        if (pathparts.length < 3 || !pathparts[1].equals("notes")) {
            this.sendEcho(req, resp);
        } else {
            this.req = req;
            // Parameter 3 is either view or inbox
            // if it is inbox, we pull in the inbox
            if (pathparts[2].equals("inbox")) {
                this.viewConfig = AppConfig.INSTANCE.getViewConfig("($Inbox)");
                this.de.execute(this);
                // if it is view we pull the respective view
            } else if (pathparts.length > 3 && pathparts[2].equals("view")) {
                this.viewConfig = AppConfig.INSTANCE.getViewConfig(pathparts[3]);
                this.de.execute(this);
            } /* more here */ else {
                // Nothing value, so we send an check only
                this.sendEcho(req, resp);
            }
        }
    }

    private void renderInbox(Session s, HttpServerResponse resp) {
        resp.headers().set("Content-Type", "application/json; charset=UTF-8");
        Database mail = s.getMailDatabase();
        resp.end(this.renderView(mail, this.viewConfig));
    }

    private void sendEcho(HttpServerRequest req, HttpServerResponse resp) {
        StringBuilder txt = new StringBuilder();
        resp.headers().set("Content-Type", "text/html; charset=UTF-8");
        txt.append("<html><body><h1>Notes request handler</h1>");
        txt.append("<h2>");
        txt.append(req.uri());
        txt.append("</h2>");
        System.out.println("Got request: " + req.uri());
        txt.append("</body></html>");
        resp.end(txt.toString());
    }

    @Override
    public boolean shouldStop() {
        // TODO Auto-generated method stub
        return false;
    }

    private String renderView(Database db, ViewConfig vc) {
        JsonBuilder b = new JsonBuilder();
        View view = db.getView(vc.getViewName());
        ViewNavigator vn = view.createViewNav();

        b.addValue("count", vn.getCount());
        b.addValue("name", vc.getViewName());
        b.startObject("documents");

        for (ViewEntry ve : vn) {
            b.startObject(ve.getUniversalID());
            b.addValue("position", ve.getPosition());
            b.addValue("isRead", ve.getRead());
            Map<String, Object> entries = ve.getColumnValuesMap();
            for (Map.Entry<String, Object> entry : entries.entrySet()) {
                String key = vc.isEmpty() ? entry.getKey() : vc.getColumnName(entry.getKey());
                if (key != null) {
                    b.addValue(key, entry.getValue());
                }
            }
            b.endObject();
        }
        b.endObject();
        return b.toString();
    }
}

08/08/2014

Designing a REST API for eMail

Category  
Unencumbered by standards designed by committees I'm musing how a REST API would look like.
A REST API consists of 3 parts: the URI (~ URL for browser access), the verb and the payload. Since I'm looking at browser only access, the structured data payload format clearly will be JSON with the prose payload delivered in MIME format. I will worry about calendar and social data later on.
The verbs in REST are defined by the HTTP standard: , PUT, and DELETE. My base url would be http://localhost:8888/email and then continue with an additional part. Combined with the 4 horsemen verbs I envision the following action matrix:

06/08/2014

Running vert.x with the OpenNTF Domino API

Category  
In the first part I got vert.x 3.0 running with my local Notes client. The mastered challenges there were 32 Bit Java for the Notes client and the usual adjustment for the path variables. The adoption of the OpenNTF Domino API required a few steps more:
  1. Set 2 evironment variables:
    DYLD_LIBRARY_PATH=/opt/ibm/notes
    LD_LIBRARY_PATH=/opt/ibm/notes
  2. Add the following parameter to your Java command line:
    -Dnotes.binary=/opt/ibm/notes -Duser.dir=/home/stw/lotus/notes/data -Djava.library.path=/opt/ibm/notes
    Make sure that it is one line only. (Of course you will adjust the path to your environment, will you?)
  3. Add 4 JAR files to the classpath of your project runtime:
    • /opt/ibm/notes/jvm/lib/ext/Notes.jar
    • /opt/ibm/notes/framework/rcp/eclipse/plugins/
      com.ibm.icu.base_3.8.1.v20080530.jar
    • org.openntf.domino.jar
    • org.openntf.formula.jar
    I used the latest build of the later two jars from Nathan's branch, so make sure you have the latest. The ICU plug-in is based on the International Components for Unicode project and might get compiled into a future version of the Domino API.
Now the real fun begins. The classic Java API is conceptually single threaded with all Domino actions wrapped into NotesThread.sinitThread(); and NotesThread.stermThread(); to gain access to the Notes C API. For external applications (the ones running neither as XPage/OSGi nor as agent, the OpenNTF API provides the Domino Executor.

17/07/2014

Adventures with vert.x, 64Bit and the IBM Notes client

Category   
The rising star of web servers currently is node.js, not at least due to the cambrian explosion in available packages with a clever package management system and the fact that "Any application that can be written in JavaScript, will eventually be written in JavaScript" (according to Jeff Atwood).
When talking to IBM Domino or IBM Connections node.js allows for very elegant solutions using the REST APIs. However when talking to a IBM Notes client, it can't do much since an external program needs to be Java or COM, the later on Windows only.
I really like node.js event driven programming model, so I looked around. In result I found vert.x, which does to the JVM, what node.js does to Google's v8 JS runtime. Wikipedia decribes vert.x as "a polyglot event-driven application framework that runs on the Java Virtual Machine ". Vert.x is now an Eclipse project.
While node.js is tied to JavaScript, vert.x is polyglot and supports Java, JavaScript, CoffeeScript, Ruby, Python and Groovy with Scala and others under consideration.
Task one I tried to complete: run a verticle that simply displays the current local Notes user name. Of course exploring new stuff comes with its own set of surprises. As time of writing the stable version of vert.x is 2.1.1 with version 3.0 under heavy development.
Following the discussion, version 3.0 would introduce quite some changes in the API, so I decided to be brave and use the 3.0 development branch to explore.
The fun part: there is not much documentation for 3.x yet, while version 2.x is well covered in various books and the online documentation.
vert.x 3.x is at the edge of new and uses Lamda expressions, so just using Notes' Java 6 runtime was not an option. The Java 8 JRE was due to be installed. Luckily that is rather easy.
The class is rather simple, even after including Notes.jar, getting it to run (more below) not so much:
package com.notessensei.vertx.notes;

import io.vertx.core.Handler;
import io.vertx.core.Vertx;
import io.vertx.core.http.HttpServerOptions;
import io.vertx.core.http.HttpServerRequest;
import io.vertx.core.http.HttpServerResponse;

import java.io.IOException;

import lotus.domino.NotesFactory;
import lotus.domino.NotesThread;
import lotus.domino.Session;

public class Demo {
	public static void main(String[] args) throws IOException {
		new Demo();
		int quit = 0;
		while (quit != 113) { // Wait for a keypress
			System.out.println("Press q<Enter> to stop the verticle");
			quit = System.in.read();
		}
		System.out.println("Veticle terminated");
		System.exit(0);
	}

	private static final int listenport = 8111;

	public Demo() {
		Vertx vertx = Vertx.factory.createVertx();
		HttpServerOptions options = new HttpServerOptions();
		options.setPort(listenport);
		vertx.createHttpServer(options)
				.requestHandler(new Handler<HttpServerRequest>() {
					public void handle(HttpServerRequest req) {
						HttpServerResponse resp = req.response();
						resp.headers().set("Content-Type",
								"text/plain; charset=UTF-8");
						StringBuilder b = new StringBuilder();
						try {
							NotesThread.sinitThread();
							Session s = NotesFactory.createSession();
							b.append(s.getUserName());
							NotesThread.stermThread();
						} catch (Exception e) {
							e.printStackTrace();
							b.append(e.getMessage());
						}
						resp.writeStringAndEnd(b.toString());
					}
				}).listen();
	}
}

Starting the verticle looked promising, but once I pointed my browser to http://localhost:8111/ the fun began.

Disclaimer

This site is in no way affiliated, endorsed, sanctioned, supported, nor enlightened by Lotus Software nor IBM Corporation. I may be an employee, but the opinions, theories, facts, etc. presented here are my own and are in now way given in any official capacity. In short, these are my words and this is my site, not IBM's - and don't even begin to think otherwise. (Disclaimer shamelessly plugged from Rocky Oliver)
© 2003 - 2017 Stephan H. Wissel - some rights reserved as listed here: Creative Commons License
Unless otherwise labeled by its originating author, the content found on this site is made available under the terms of an Attribution/NonCommercial/ShareAlike Creative Commons License, with the exception that no rights are granted -- since they are not mine to grant -- in any logo, graphic design, trademarks or trade names of any type. Code samples and code downloads on this site are, unless otherwise labeled, made available under an Apache 2.0 license. Other license models are available on written request and written confirmation.