Search

Domino Upgrade

VersionSupport end
5.0
6.0
6.5
7.0
8.0
8.5
Upgrade to 9.x now!
(see the full Lotus lifcyle) To make your upgrade a success use the Upgrade Cheat Sheet.
Contemplating to replace Notes? You have to read this! (also available on Slideshare)

Languages

Other languages on request.

Twitter

Useful Tools

Get Firefox
Use OpenDNS
The support for Windows XP has come to an end . Time to consider an alternative to move on.

About Me

I am the "IBM Collaboration & Productivity Advisor" for IBM Asia Pacific. I'm based in Singapore.
Reach out to me via:
Follow notessensei on Twitter
(posts)
Skype
Sametime
IBM
Facebook
LinkedIn
XING
Amazon Store
Amazon Kindle
NotesSensei's Spreadshirt shop
profile for stwissel on Stack Exchange, a network of free, community-driven Q&A sites

21/07/2017

From Blogsphere to a Static Site (Part 5) - Comment front-end

Category

In Part 4 I described the comment backend. This installment sheds a light on the comment front-end.

Comments can be tricky. One lesson I learned early: When your comment form is standard HTML form, it attracts spam like a light bulb attracts moths. So the requirement were:

  • The original blog entry should not contain any HTML form. It should be loaded on a button click using JavaScript. Nota bene: this isn't hide/show, but actual manipulation of the DOM
  • The dynamic form shall not contain a POST URL, but submission should be in JavaScript - keeps a lot of the scumbags out already
  • Submission should be secured with a Captcha
  • Some formatting should be allowed. I opted for a Markdown editor with preview capabilities

The first component is the placeholder for the existing comments and the button showing the comment form:

<a name="comments"></a>
{{^commentsclosed}}
<div class="well well-raised" style="text-align : center">
  <button class="btn btn-lg btn-info" data-toggle="collapse" data-target="#commentform_{{UNID}}" type="button">
    Add your comment...  <span class="glyphicon glyphicon-comment"></span>
  </button>
</div>
<div id="commentform_{{UNID}}" class="collapse"></div>
{{/commentsclosed}}
<div class="well well-raised">
  <h4>Comments</h4>
  <ol id="commentList">
    {{#comments}}
    <li>
      {{#gravatarURL}}<img src="{{.}}" class="gravatarimg" /> {{/gravatarURL}} posted by <b>{{author}}</b> on <i>{{createdString}}</i>:
      <br /> {{& comment}}
      <hr style="clear : both" />
    </li> {{/comments}} {{^comments}}
    <li id="nocomments">
      <h5>No comments yet, be the first to comment</h5>
    </li>
    {{/comments}}
  </ol>
</div>

The second component is the comment form, implemented as mustache template - one of the reasons I picked Mustache: runs on the server and the client in tons of languages

<form title="Comment form for blog discussion" onSubmit="return addComment(this,'{{recaptchaid}}','{{parentId}}')" class="form-vertical well well-raised">
  <fieldset>
    <legend>Add your comment</legend>
    <p>Please note: <b>Comments without a valid and working eMail address will be removed.</b>
      <br /> This is my site, so I decide what stays here and what goes.</p>
     <div class="control-group" id="commentcontrol">
      <label class="control-label" for="Commentor">Name (required, published)</label>
      <div class="controls">
        <input class="input-xlarge focused" id="Commentor" size="30" accesskey="n" name="nameAuthor" />
      </div>
      <label class="control-label" for="Email">eMail (required, not published)</label>
      <div class="controls">
        <input type="eMail" class="input-xlarge focused" id="Email" size="30" accesskey="n" name="txtEmail" placeholder="A working eMail please!" />
      </div>
      <label class="control-label" for="webSite">URL (optional)</label>
      <div class="controls">
        <input type="url" class="input-xlarge" id="webSite" size="30" accesskey="n" name="txtWebSite" />
      </div>
      <div class="controls">
        <div id="wmd-panel" class="wmd-panel">
          <table style="width : 100%" border="0">
            <tr>
              <td style="width : 50%; vertical-align : top">
                <label class="control-label" for="wmd-input">Your Comment (Use markdown like <a href="//stackoverflow.com/editing-help" target="_blank">Stackoverflow</a>)</label>
                <div id="wmd-button-bar"></div>
                <textarea class="wmd-input" id="wmd-input" name="Body"></textarea>
              </td>
              <td style="width : 50%; vertical-align : top">
                <label class="control-label">Preview</label>
                <div id="wmd-preview" class="wmd-panel wmd-preview"></div>
              </td>
            </tr>
          </table>
        </div>
      </div>
      <div class="controls" id="captchadiv">Captcha here</div>
      <div class="form-actions">
        <button id="commentsubmit" type="submit" class="btn btn-primary btn-large">Post your comment</button>
      </div>
    </div>
    <div class="alert alert-block" id="alertContainer" style="display : none">One moment please, submitting comment...</div>
  </fieldset>
</form>

The whole mechanism gets to work with just a few jQuery JavaScript functions (Vanilla JS would work too, but I had JQuery already for the social buttons, so I reused that) and the respective JS files:

04/05/2017

From Blogsphere to a Static Site (Part 4) - Comment backend

Category

The blog needed a comment function. While there are social options around (Facebook, Disqus etc), I decided I want to roll my own. Partly because I want tighter control and partly, well, because I could. My comment backend would:

  • Provide a REST API to create comments in a JSON structure. The comment body will be Markdown. Reading would provide comments in ready to use HTML (I hear howling from the API crowd). No delete or update functionality
  • Cleanup content considered harmful (code injection) and optional sport Captcha
  • Store all content in a NoSQL database, in my case CouchDB (or Cloudant with its 20G free plan)
  • Cache all queries for comment in an online cache to limit calls to the database
  • Initially run on Domino, later on liberty or the raw JVM
  • Initially also update Domino using a web service - so during transition no comments would get lost

In its initial incarnation the Comment servlet is a OSGi plugin that listens to the /comments URL implemented as Wink servlet. So the class of interest is the one defining the service. We have one method for post, one for get and a helper function

/**
 * Wink implementation of Comment service
 */
@Workspace(workspaceTitle = "Blog Comments", collectionTitle = "Create or display comments")
@Path(value = "/comments")
@Produces(MediaType.TEXT_HTML)
@Consumes(MediaType.APPLICATION_JSON)
public class CommentService extends CommentResponse {

    private final Logger logger = Logger.getLogger(this.getClass().getName());

    @POST
    public Response createComment(@Context HttpServletRequest request) {
        final Monitor mon = MonitorFactory.start("CommentService#createComment");
        String result = "Sorry I can't process your comment at this time";
        ResponseBuilder builder = Response.ok();

        try {
            InputStream in = request.getInputStream();
            BlogComment comment = BlogComment.load(in);
            in.close();

            if (comment != null) {
                this.captureSubmissionDetails(request, comment);
                result = CommentManager.INSTANCE.saveComment(comment, true);
            } else {
                builder.status(Status.NOT_ACCEPTABLE);
            }
            builder.entity(result).type(MediaType.TEXT_HTML_TYPE);

        } catch (Exception e) {
            String errorMessage = e.getMessage();
            builder.entity((((errorMessage == null) || errorMessage.equals("")) ? "Undefined error" : errorMessage)).type(
                    MediaType.TEXT_HTML_TYPE);
            Utils.logError(this.logger, e);
        }

        mon.stop();
        return builder.build();
    }

    @GET
    public Response getComments(@QueryParam("parentid") final String parentid) {
        Response response = null;
        final Monitor mon = MonitorFactory.start("CommentService#getComments");
        final ResponseBuilder builder = Response.ok();
        final Collection<BlogComment> bc = CommentManager.INSTANCE.loadComments(parentid);

        if ((bc == null) || bc.isEmpty()) {
            builder.status(Status.NO_CONTENT);
        } else {
            response = this.renderOutput(bc, "comment.mustache");
        }
        mon.stop();
        return (response == null) ? builder.build() : response;
    }

    private void captureSubmissionDetails(HttpServletRequest request, BlogComment comment) {
        final Monitor mon = MonitorFactory.start("CommentService#captureSubmissionDetails");
        try {
            @SuppressWarnings("rawtypes")
            Enumeration hn = request.getHeaderNames();
            if (hn != null) {
                while (hn.hasMoreElements()) {
                    String key = hn.nextElement().toString();
                    comment.addParameter(key, request.getHeader(key));
                }
            }
            @SuppressWarnings("rawtypes")
            Enumeration pn = request.getParameterNames();
            if (pn != null) {
                while (pn.hasMoreElements()) {
                    String key = pn.nextElement().toString();
                    String[] values = request.getParameterValues(key);
                    comment.addParameters(key, values);

                    if (key.equals("referer")) {
                        comment.setReferer(values[0]);
                    } else if (key.equals("user-agent")) {
                        comment.setUserAgent(values[0]);
                    }

                }
            }
            @SuppressWarnings("rawtypes")
            Enumeration an = request.getAttributeNames();
            if (an != null) {
                while (an.hasMoreElements()) {
                    try {
                        String key = an.nextElement().toString();
                        comment.addAttribute(key, String.valueOf(request.getAttribute(key)));
                    } catch (Exception e) {
                        // No action here
                    }
                }
            }
            comment.addParameter("REMOTE_HOST", request.getRemoteHost());
            comment.addParameter("REMOTE_ADDR", request.getRemoteAddr());
            comment.addParameter("REMOTE_USER", request.getRemoteUser());

            // Needed for Captcha
            comment.setRemoteAddress(request.getRemoteAddr());
        } catch (Exception e) {
            Utils.logError(this.logger, e);
            // But no further action here!
        }
        mon.stop();
    }
}

01/05/2017

From Blogsphere to a Static Site (Part 3) - Generating pages

Category

The rendering engine I choose is mustache which describes itself as "logic-less templating engine". My main criteria was the availability on multiple platforms including Java and JavaScript (I might port the rendering part to NodeJS at some time in the future).
The only logic mustache supports is conditional rendering based on the presence or absence of an element. When an element is present and is an array (or a collection in Java) the body of the template gets repeated for each element in the array. A scalar value hence is treated as an array with one value only.

Mustache is simple to use. All you need is a data bean (in Java, a JSON structure for JavaScript) and a text file containing placeholders with the property names. E.g. <h1>{{title}}</h1> will render a headline with the title property of you data object. In Java that would be either a public variable or a call to getTitle according to the bean specification. The blog rendering code therefore is quite simple:

private void renderOneEntry(BlogEntry be, Mustache mustache) throws IOException {

        String location = this.config.destinationDirectory + be.getNewURL();
        String outDirs = location.substring(0, location.lastIndexOf("/"));
        File dirs = new File(outDirs);
        if (!dirs.exists()) {
            dirs.mkdirs();
        }
        // Set the current context
        for (LinkItem cat : be.getCategory()) {
            String c = cat.name;
            this.allCategories.get(c.toLowerCase()).active = true;
        }
        this.allDateCategories.get(be.getDateYear()).active = true;

        if (be.getSeries() != null) {
            String series = be.getSeries();
            if (this.allSeries.containsKey(series)) {
                this.allSeries.get(series).get(be.getNewURL()).active = true;
            }
        }

        // Prepare to write out
        ByteArrayOutputStream out = new ByteArrayOutputStream(102400);
        Writer pw = new PrintWriter(out);

        // This is where the magic happens
        mustache.execute(pw, be);
        pw.flush();
        pw.close();
        this.saveIfChanged(out.toByteArray(), location);

        // Cleanup
        for (LinkItem cat : be.getCategory()) {
            String c = cat.name;
            this.allCategories.get(c.toLowerCase()).active = false;
        }
        this.allDateCategories.get(be.getDateYear()).active = false;

        if (be.getSeries() != null) {
            String series = be.getSeries();
            if (this.allSeries.containsKey(series)) {
                this.allSeries.get(series).get(be.getNewURL()).active = false;
            }
        }
    }

The actual rendering is just the line mustache.execute(pw, be); The code around it prepares and resets the collections that might render on a page like categories, series or month and year. Also of interest is this.saveIfChanged(out.toByteArray(), location); which only saves results back to disk if it actually has changed. Don't be mistaken: any change in layout will lead to a newly rendered page, so this is quite important to save as needed and not more (you don't want to have tons of identical files that only differ in their time stamp)

17/04/2017

From Blogsphere to a Static Site (Part 2) - Cleaning up the HTML

Category

Blogsphere allows to create RichText and plain HTML entries. To export them I need to grab the HTML, either the manual entered or the RichText generated on, clean it up (especially for my manual entered HTML) and then replace image sources and internal links using the new URL syntax. To make this happen I created 2 functions that saved images and attachments and created a lookup list, so the HTML cleanup has a mapping table to work with

private void saveImage(Document doc) {
        String sourceDirectory = this.config.sourceDirectory + this.config.imageDirectory;
        try {
            String subject = doc.getItemValueString("ImageName");
            Date created = doc.getCreated().toJavaDate();
            @SuppressWarnings("rawtypes")
            Vector attNames = this.s.evaluate("@AttachmentNames", doc);
            String description = doc.getItemValueString("ImageName");
            String oldURL = this.config.oldImageLocation + doc.getItemValueString("ImageUNID") + "/$File/";
            SimpleDateFormat sdf = new SimpleDateFormat("yyyy");
            String year = sdf.format(created);
            FileEntry fe = this.imgEntries.add(subject, oldURL, description, created);

            for (Object attObj : attNames) {
                try {
                    String attName = attObj.toString();
                    String newURL = this.config.webBlogLocation + this.config.imageDirectory + year + "/" + attName;
                    fe.add(attName, newURL, description, created);
                    String outDir = sourceDirectory + year + "/";
                    this.ensureDirectory(outDir);
                    EmbeddedObject att = doc.getAttachment(attName);
                    att.extractFile(outDir + attName);
                    Utils.shred(att);
                } catch (NotesException e) {
                    e.printStackTrace();
                } catch (Exception e2) {
                    e2.printStackTrace();
                }
            }

        } catch (NotesException e) {
            e.printStackTrace();
        }

    }

    private void saveImageFromURL(String href, String targetName) {

        String fetchFromWhere = "https://" + this.config.bloghost + href;
        try {
            byte[] curImg = Request.Get(fetchFromWhere).execute().returnContent().asBytes();
            this.saveIfChanged(curImg, targetName);
        } catch (ClientProtocolException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

    }

With images saved the HTML cleanup can proceed. As mentioned before I'm using JSoup to process crappy HTML. It allows for easy extraction of elements and attributes, so processing of links an images is just a few lines

15/04/2017

From Blogsphere to a Static Site (Part 1) - Objects

Category
The migration tooling revolves around data, so getting the data classes right is important. The data classes need to be designed to allow it to be populated either from the Blogsphere NSF or from a collection of JSON files (so the blog generation can continue when the NSF is gone). For the blog we need 3 objects:
  • BlogEntry: The main data containing a blog entry including its meta data
  • BlogComment: An entry with a comment for a Blog in a 1:n relation
  • FileEntry: information about downloadable files (needed for export)

There will be auxiliary data classes like Config, RenderInstructions, Blogindex. Their content is derived from the data stored in the main object or, in case of Config, from disk.

Data classes in the Blog

13/04/2017

From Blogsphere to a Static Site (Part 0) - Requirements

Category
Readers of this blog might have noticed, that blog layout and blog URLs have changed (a while ago). This blog now serves static HTML pages using a nginx web server (get used to nginx, it's coming in Connections Pink too). I will document the steps and code I used to get there. Step 0 is: define the requirements and evaluate the resulting constraints:
  • Export of all Blogsphere content to HTML, including the conversion of entries written in RichText
  • No export of configuration or layout required
  • New site structure that shows articles in year and month folders
  • Modular templating system with includes for repeated pieces (e.g. header, footer, sidebar)
  • Summary pages for year, month and categories
  • Summary page for article series
  • Existing comments to become part of the html page
  • Repeatability: export must be able to repeat, but not overwrite a page that hasn't actually changed
  • Storage of exported pages in a file structure as JSON files
  • Rendering of static site from Notes or from JSON directory
  • Redirection file, so old links get a proper redirection to the new URL
  • Have a comment database for new comments
  • No pagination for any of the summary pages (I might change my mind on that one)
  • Cleanup messy HTML automatically, fix syntax and URLs to posts and images
  • Optimized HTML, CSS and JS for speedy delivery
I had a look at Jekyll, which is the templating engine GitHub is using. I would have allowed me to just commit a new file and Github would render for me. Unfortunately Jekyll fell short of the article series and category overview pages.

12/06/2008

500

Category
My first blog entry on 01 Jan 2003 was titled Predictions for 2003. I started blogging out of curiosity. Finding my place in the "Speakers Corner of the World" was an interesting journey. Over the last 5 years I added 500 entries to this blog. You are reading entry #500. The frequency of entries varied based on workload, mood or the fact that I had something or nothing to say.
I owe my customers a lot of the ideas I blogged about in the "Show-and-Tell" and "Lotus Notes" categories. Without their real world problems I wouldn't had the need to think hard. I owe David Ferris the idea to blog about What are the strongest aspects of Lotus Domino?. That entry earned me a call from IBM in Singapore and let to my current job as "Lotus Technology & Productivity Advisor", LTPA for short.
Over time my topics shifted from more personal towards more professional stuff. Seems my family is taking over to report the private stuff. I already have more than a dozend draft entries and even more topics around Notes and Domino waiting to be written, so I guess I'll stay on the professional trajectory. I'm curious if it will take another 5 years to reach 1000. We will see.
Stay tuned.

10/04/2008

Creating SQL Statements from form definitions

QuickImage Category  
I had an iteresting discussion with a customer this week. They use Domino and dotNet for their web applications. Their decission criteria when to use what: if the data of the application needs to be fed into their data warehouse at a later point of time, they use dotNet since storage there typically ends up in an RDBMS. The biggest problem they face, in their own voice: "Our users are pretty spoiled from Domino. They expect days as turnaround time for applications. Using dotNet it takes at least three times longer."
So I asked why they don't use DECS to connect to the RDBMS. They could develop the application in Notes/Domino and once the app does what the user wants just add the tables in the RDBMS and link them up using DECS. They asked back if there is a way to generate the table or at least the create table statement from Domino directly. The short answer: Yes, you can, however you need to make decissions on datatypes and field length. The long answer: you need Domino Designer (for the Tools - DXL Utilities - Transformer ... menu) and a little XSLT stylesheet.

12/12/2007

Back online -- sort of

Category
The Blog upgrade went smooth. Replace the design, open the database, say yes to the "Upgrade to V3" question, copy the layouts and update the configuration. I only hit one problem: ADSL. Uploading is much slower that downloading, so all the update/upgrades are only trickling back to the server (I had about 200k hit documents still in the database). So it might be a while before I will look for a new layout.

11/12/2007

Blog disruptions expected

Category
I'll be upgrading to Blogsphere V3. I push it to the test: just replace the design and see what is happening. Blog service might be disrupted. stw

Disclaimer

This site is in no way affiliated, endorsed, sanctioned, supported, nor enlightened by Lotus Software nor IBM Corporation. I may be an employee, but the opinions, theories, facts, etc. presented here are my own and are in now way given in any official capacity. In short, these are my words and this is my site, not IBM's - and don't even begin to think otherwise. (Disclaimer shamelessly plugged from Rocky Oliver)
© 2003 - 2017 Stephan H. Wissel - some rights reserved as listed here: Creative Commons License
Unless otherwise labeled by its originating author, the content found on this site is made available under the terms of an Attribution/NonCommercial/ShareAlike Creative Commons License, with the exception that no rights are granted -- since they are not mine to grant -- in any logo, graphic design, trademarks or trade names of any type. Code samples and code downloads on this site are, unless otherwise labeled, made available under an Apache 2.0 license. Other license models are available on written request and written confirmation.