Posts tagged "workflow"

May 19, 2012

How to downsize OOTB DAM’s web rendition

By default, upon image upload to CQ’s Digital Asset Management, a “DAM Update Asset” workflow would be triggered and one of the many processes inside the workflow is to generate a web rendition of the uploaded image. And the default setting of CQ’s Image API is to always render the web-enabled version of the uploaded image. This combination helps to limit the size and the quality of the image displayed thus reducing the page load time.

The OOTB setting for the web-enabled image dimensions is set to 1280 pixels by 1280 pixels. And the image quality is set to reduce to 90%. To change this setting, browse to CQ’s workflow console and select the “DAM Update Asset” workflow under the Models tab:

Inside the workflow model, select the Web enabled rendition step:

Then adjust the settings accordingly. In my example below, I have set the maximum dimension to be 640pixel by 640pixel. I have also reduced the quality of the images to 80%:

 

Preliminary testing shows that file sizes of this setting (in comparison with the OOTB 1280px by 1280px; 90% quality setting) is reduced by a third, while difference in image quality is minimal. YMMV, please adjust your settings accordingly.

12:36 PM Permalink
February 25, 2012

How to integrate CQ5 with Defensio Profanity detection

I have recently worked on a project where Defensio integration is needed to detect profanity in CQ5′s commenting system and discussion forums. Adobe CQ5 comes with OOTB integration with Automattic Kismet (Akismet) which is for Spam filtering but Akismet does not offer profanity filtering. For more information on Adobe CQ5′s integration with Akismet, please look at the following links:

Social Collaboration – Setting an Akismet Key

AkismetService API doc

Defensio is a profanity and spam filtering service on user generated content. It offers a fully RESTful API to automatically detect, reject, and mask profanity from user generated content on a website. To learn more about Defensio, please visit:

Defensio Web Service – Powered by WebSense

Defensio API Specification

 

Now let’s talk about the high level requirements of the integration I have recently worked on, before I go into the technical details.

Requirements:

  • Profanity detection should be enabled for any comments or discussion forum replies (user generated content).
  • If obscene words are found, they should be masked and the comment / forum reply should be marked as spam. The spam message should not be displayed unless approved by a soco (social collaboration) admin user.
  • No profanity messages should be displayed publicly. If it should be displayed (approved by admin), the masked version is used.
  • The Defensio Web Service API URL and also the API key should be configurable.

Technical Details:

  • A workflow process (com.day.cq.workflow.exec.WorkflowProcess) should be created for the profanity check so that it can be enabled in any CQ workflows.
  • To enable profanity detection for comments or discussion forum replies, the workflow process can be inserted into the OOTB “Comment Moderation” workflow.
  • In order to allow system administrators to configure the Defensio Web Service URL and the API key, an OSGi bundle can be created with service component runtime properties like the following:

 

OSGi Bundle implementation:

This blog entry is not to discuss the very detail of how to package and deploy an OSGi Bundle into Apache Felix. The code snippets I’m showing below utilized the Apache Maven software project management tool, and the codes are greatly shorted (life is easy with Java Annotations) using the maven-bundle-plugin and maven-scr-plugin. To learn more about these plugins, here are some links:

Apache Felix Bundle Plugin for Maven

Apache Felix Maven SCR Plugin

The OSGi bundle for the Defensio integration consists of three classes:

com.customername.cq.profanity.DefensioService
com.customername.cq.profanity.DefensioServiceImpl
com.customername.cq.profanity.DefensioProcess

DefensioService is the interface for the Defensio Profanity integration,
DefensioServiceImpl is an implementation of the integration, and
DefensioProcess is a CQ workflow process implementation (makes it available as a process step in a CQ workflow).

So here you go, please note the use of annotations, and also note that some of logs and minor implementation details are skipped (marked by ellipses in comments):

com.customername.cq.profanity.DefensioService:
package com.customername.cq.profanity;
 
public interface DefensioService {
 
    /**
     * Check to see if the Defensio API key has been verified
     *
     * @return <code>true</code> if the API key has been verified, <code>false</code> otherwise
     */	
	public boolean isVerifiedKey();
 
    /**
     * Verify Defensio API key
     *
     * @return <code>true</code> if the API key has been verified, <code>false</code> otherwise
     */	
	public boolean verifyAPIKey();
 
	@SuppressWarnings("unchecked")
	public String profanityCheck(String commentContent);
}
com.customername.cq.profanity.DefensioServiceImpl:
package com.customername.cq.profanity;
 
import java.io.IOException;
import java.util.Dictionary;
 
import org.apache.commons.httpclient.DefaultHttpMethodRetryHandler;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.params.HttpClientParams;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.sling.commons.json.JSONException;
import org.apache.sling.commons.json.JSONObject;
import org.osgi.framework.BundleContext;
import org.osgi.service.component.ComponentContext;
 
/**
 * Defensio Service implementation
 * <p></p>
 * <a href="http://http://www.defensio.com/api">Defensio API</a> documentation.
 *
 * @author Oliver Choy
 *
 * @scr.component label="Adobe CQ AntiProfanity"
 *                description="Defensio configuration"
 * @scr.property name="service.vendor" value="Customer Name"
 * @scr.property name="service.description" value="Defensio Profanity Check for blog comments"
 * @scr.service
 */
public class DefensioServiceImpl implements DefensioService {
 
    private Log logger = LogFactory.getLog(DefensioServiceImpl.class);
 
    // Constants
    private static final String USER_AGENT_HEADER = "User-Agent";
    private static final String USER_AGENT_VALUE = "Adobe CQ5";
    private static final String API_PARAMETER_COMMENT_CONTENT = "comment_content";
 
    private HttpClient httpClient;
    private String apiKey;
    private boolean verifiedKey = false;
    private String providerUrlFragment;
    
    /** @scr.property */
    public static final String PARAM_API_KEY = "defensio.service.api.key";
    /** @scr.property */
    public static final String PARAM_API_URL = "defensio.service.api.url";
 
    public DefensioServiceImpl() {
    }
 
    DefensioServiceImpl(BundleContext bundleContext, Dictionary<String, Object> configuration) {
        this.setup(bundleContext, configuration);
    }
 
    void setup(BundleContext bundeContext, Dictionary<String, Object> configuration) {
    	// get api key
        Object key = configuration.get(PARAM_API_KEY);
        if (key != null) {
            this.apiKey = key.toString();
        }
        // get api url
        Object apiUrl = configuration.get(PARAM_API_URL);
        if (apiUrl != null) {
            this.providerUrlFragment = apiUrl.toString();
        }
 
        // autoverify
        if(providerUrlFragment != null && apiKey != null) {
            if(providerUrlFragment.length() > 0 && apiKey.length() > 0) {
        	logger.info("Defensio API key verification result is " + verifyAPIKey());
            }
        }
    }
 
    void shutdown() {
    	this.apiKey = null;
    }
 
    @SuppressWarnings("unchecked")
    protected void activate(ComponentContext context) {
	httpClient = new HttpClient();
	HttpClientParams httpClientParams = new HttpClientParams();
	DefaultHttpMethodRetryHandler defaultHttpMethodRetryHandler = new DefaultHttpMethodRetryHandler(0, false);
	httpClientParams.setParameter(USER_AGENT_HEADER, USER_AGENT_VALUE);
	httpClientParams.setParameter(HttpClientParams.RETRY_HANDLER,
		defaultHttpMethodRetryHandler);
	httpClient.setParams(httpClientParams);
	setup(context.getBundleContext(), context.getProperties());
    }
 
    protected void deactivate(ComponentContext context) {
        this.shutdown();
    }
 
    public boolean verifyAPIKey() {
    	boolean callResult = false;
        String defensioUserURL = "http://" + providerUrlFragment + apiKey + ".json";
        GetMethod get = new GetMethod(defensioUserURL);
 
        try {
            httpClient.executeMethod(get);
            String result = get.getResponseBodyAsString();
            
            if (result != null && !result.equals("")) {
            	JSONObject respObj = new JSONObject(result);
            	JSONObject defensioResultObj = respObj.getJSONObject("defensio-result");
            	if (defensioResultObj != null) {
            	    String verifyStatus = defensioResultObj.getString("status");
                    if (verifyStatus.equals("success")) {
            		callResult = true;
            	    }
            	} 
            }
        } catch (IOException e) {
            logger.error("...");
        } catch (JSONException e) {
      	    logger.error("...");
	}
 
        verifiedKey = callResult;
        return callResult;
    }
 
    /**
     * Generic call to Defensio
     **/
    @SuppressWarnings("unchecked")
	private String defensioCall(String commentContent) {
    	
    	String filteredResult = "";
        String defensioProfanityURL = "http://" + providerUrlFragment + apiKey + "/profanity-filter.json";
 
        PostMethod post = new PostMethod(defensioProfanityURL);
        post.addRequestHeader("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");
 
        if (commentContent != null) {
            post.addParameter(new NameValuePair(API_PARAMETER_COMMENT_CONTENT, commentContent));
        }
 
        try {
            httpClient.executeMethod(post);
            String result = post.getResponseBodyAsString();
 
            if (result != null && !result.equals("")) {
            	JSONObject respObj = new JSONObject(result);
            	JSONObject defensioResultObj = respObj.getJSONObject("defensio-result");
            	if (defensioResultObj != null) {
            	    String verifyStatus = defensioResultObj.getString("status");
            	    if (verifyStatus.equals("success")) {
            		JSONObject filteredObject = defensioResultObj.getJSONObject("filtered");
            		if (filteredObject != null) {
            		    filteredResult = filteredObject.getString(API_PARAMETER_COMMENT_CONTENT);
            		    logger.debug("Defensio filtered comment: " + filteredResult);
            		}
            	    }
            	} 
            }
        } catch (IOException e) {
            logger.error("...");
        } catch (JSONException e) {
       	    logger.error("...");
	}
        return filteredResult;
    }
 
    @SuppressWarnings("unchecked")
	public String profanityCheck(String commentContent) {
        return defensioCall(commentContent);
    }
    
    public boolean isVerifiedKey() {
        return verifiedKey;
    }
}
com.customername.cq.profanity.DefensioProcess:
package com.customername.cq.profanity;
 
import javax.jcr.Node;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
 
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.jcr.resource.JcrResourceResolverFactory;
import org.osgi.service.component.ComponentContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
 
import com.day.cq.workflow.WorkflowException;
import com.day.cq.workflow.WorkflowSession;
import com.day.cq.workflow.exec.WorkItem;
import com.day.cq.workflow.exec.WorkflowData;
import com.day.cq.workflow.exec.WorkflowProcess;
import com.day.cq.workflow.metadata.MetaDataMap;
 
/**
 * This <code>JavaProcess</code> checks a comment or forum reply for profanity
 * 
 * @scr.component metatype="false"
 * @scr.service interface="com.day.cq.workflow.exec.WorkflowProcess"
 * @scr.property name="process.label" value="Defensio Profanity Check"
 */
public class DefensioProcess implements WorkflowProcess {
 
    private static final String COMMENT_RESOURCE_TYPE = "collab/commons/components/comments/comment";
    private static final String FORUM_POST_RESOURCE_TYPE = "collab/forum/components/post";
    private static final String FORUM_TOPIC_RESOURCE_TYPE = "collab/forum/components/topic"
    private static final String SLING_RESOURCE_TYPE = "sling:resourceType";
    private static final String TYPE_JCR_PATH = "JCR_PATH";
    private final Logger logger = LoggerFactory.getLogger(this.getClass().getName());
 
    /**
     * @scr.reference policy="static"
     */
    private DefensioService defensioService;
    /**
     * @scr.reference policy="static"
     */
    private JcrResourceResolverFactory jcrResourceResolverFactory;
 
    protected void activate(final ComponentContext context) {
        // check if the key is valid
        logger.info("Verifying key for Defensio check");
        boolean isValidKey = defensioService.isVerifiedKey();
        if (isValidKey) {
            logger.info("...");
        } else {
            logger.warn("...");
        }
    }
 
    /**
     * @see WorkflowProcess#execute(WorkItem, WorkflowSession, MetaDataMap)
     */
    public void execute(WorkItem workItem, WorkflowSession workflowSession, MetaDataMap args) throws WorkflowException {
        final Session session = workflowSession.getSession();
        final WorkflowData data = workItem.getWorkflowData();
        String path = null;
        String type = data.getPayloadType();
        try {
            if (type.equals(TYPE_JCR_PATH) && data.getPayload() != null) {
                String payloadData = (String) data.getPayload();
                if (session.itemExists(payloadData)) {
                    path = payloadData;
                }
            }
 
            if (path != null) {
                final Node userGeneratedNode = (Node) session.getItem(path);
 
                if (userGeneratedNode.hasProperty(SLING_RESOURCE_TYPE) &&
                        (userGeneratedNode.getProperty(SLING_RESOURCE_TYPE).getString().equals(COMMENT_RESOURCE_TYPE) ||
                        	userGeneratedNode.getProperty(SLING_RESOURCE_TYPE).getString().equals(FORUM_POST_RESOURCE_TYPE) || 
                                userGeneratedNode.getProperty(SLING_RESOURCE_TYPE).getString().equals(FORUM_TOPIC_RESOURCE_TYPE))
                ) {
 
                    logger.debug("Profanity check for comment node at " + userGeneratedNode.getPath());
                    String ipAddress = null;
                    if (userGeneratedNode.hasProperty("ip")) {
                        ipAddress = userGeneratedNode.getProperty("ip").getString();
                    } else {
                        setIsSpam(userGeneratedNode, false, session);
                    }
                    // if the ip is localhost we set isspam to false as well
                    if (ipAddress.equalsIgnoreCase("127.0.0.1") || ipAddress.equalsIgnoreCase("localhost")
                            || ipAddress.equalsIgnoreCase("0:0:0:0:0:0:0:1")) {
                        setIsSpam(userGeneratedNode, false, session);
                    }
 
                    // UserAgent spam check ...
                    // referrer spam check ...
                    // Author spam check ...
                    // author email spam check ...
                    // author URL spam check ...
                    
                    String commentText = null;
                    if (userGeneratedNode.hasProperty("jcr:description")) {
                        commentText = userGeneratedNode.getProperty("jcr:description").getString();
                    }
 
                    String filteredText = defensioService.profanityCheck(commentText);
                    if (!filteredText.equals(commentText)) {
                    	setIsSpam(userGeneratedNode, true, session);
                    	setFilteredText(userGeneratedNode, filteredText, session);
                    }
                    logger.info("Filtered result " + filteredText + " for comment: " + workItem.toString());
 
                } else {
                    logger.warn("Cannot check for profanity because item is not a comment/topic/post. Workitem: "
                            + workItem.toString());
                }
            } else {
                logger.warn("Cannot check for spam because path is null for this workitem: " + workItem.toString());
            }
        } catch (RepositoryException e) {
            throw new WorkflowException(e);
        }
    }
 
    private void setFilteredText(Node commentNode, String value, Session session) throws WorkflowException {
        try {
            commentNode.setProperty("jcr:description", value);
            session.save();
        } catch (Exception e) {
            throw new WorkflowException(e);
        }
    }
    
    private void setIsSpam(Node commentNode, boolean value, Session session) throws WorkflowException {
        try {
            commentNode.setProperty("isSpam", value);
            session.save();
        } catch (Exception e) {
            throw new WorkflowException(e);
        }
    }
}

Workflow Configuration:

Once the OSGi bundle is implemented and installed, a Process Step can be added to the CQ comment moderation workflow to trigger the profanity check.

Note that the Process Step, once dragged into the workflow, will need to be configured to use the OSGi bundle we have installed. In the example above, the Process Step is named as “Check Profanity” and the process chosen is “Defensio Profanity Check”, like the following:

 

Profanity Detection Result:

Once all of the above are implemented, installed, and configured, you can test the comment entry and discussion forum reply in CQ. It should yield something similar to the following:

The above image depicts two commenting scenarios (in CQ author environment): The first entry without profanity was successfully inserted into the system; the second comment was masked due to discovery of profanity, and was marked as spam waiting for approval/denial of a social collaboration administrator.

1:20 PM Permalink