Working with Layers

Layers, in graphic arts programs, give you a way of organizing objects in your documents and controlling their front-to-back stacking order. Illustrator, InDesign, and Photoshop all have layers, and share most basic layer features: you can create, move, hide, and show layers, and you can assign objects to layers. Illustrator and Photoshop have the ability to create layers within layers—“layer sets” or “layer groups.”

As is usual, the scripting model differs among the applications, so I’ll provide a set of wrapper functions that will take the same parameters, regardless of the host application. I’ll put all of the application-specific details inside these functions. I’ll make the functions work for Photoshop layer groups and Illustrator sub-layers, as well as for InDesign’s simpler layers. I don’t want to rebuild each application’s Layers panels, so I’ll provide simple buttons for putting the layer functions through their paces. (It seems to me unlikely that you’ll want to re-create the Layers panel, and more likely that you’ll want to add/assign/move layers without displaying a user interface at all. If I’m dead wrong, please let me know!)

This example is even more basic than the previous one—but I found a number of points that might trip up developers trying to work with layers in their extensions.

You can find the example project here:

layers

Setting Up

If you’ve been following this series of posts, you know the drill: I’ll create the project using the Creative Suite Extension Builder tooling, then I’ll add the AppController class (for non-application-specific code), the AppModel class (for data storage), and the DocumentWatcher class (which adds event listeners for document open/close events). All of the application-specific code will go into classes corresponding to the host applications that the extension supports (layersIlllustrator.as, layersInDesign.as, and layersPhotoshop.as). We won’t need to add the host adapters to catch application-specific events this time—monitoring CSXS document events alone will do the trick.

The user interface (in main.mxml) for this example is about as simple as can be: buttons that perform layer-related operations. One of the buttons creates an example document; the others are not available unless that document (or, really, any instance of that document—we’re not too discriminating) is open.

main.mxml

<?xml version="1.0" encoding="utf-8"?>
<csxs:CSXSWindowedApplication
	xmlns:csxs="com.adobe.csxs.core.*" 
	xmlns:mx="http://www.adobe.com/2006/mxml" 
	layout="absolute" 
	historyManagementEnabled="false" 
	creationComplete="onCreationComplete()"
	applicationComplete="onApplicationComplete()" 
	close="onClose()"
	>
	<mx:Script>
		<![CDATA[
			import layers.layersIllustrator;
			import layers.layersInDesign;
			import layers.layersPhotoshop;
			import layers.AppController;
			import layers.AppModel;
			import layers.DocumentWatcher;			
			[Bindable]
			private var model:AppModel = null; 
			private var controller:AppController = null;
			private var watcher:DocumentWatcher = null;
			private var hostName:String = HostObject.mainExtension;
			private function onCreationComplete():void
			{
				model = AppModel.getInstance();				
				model.readyState();
			}
			private function onApplicationComplete():void
			{
				controller = AppController.getInstance();
				watcher = DocumentWatcher.getInstance();
			}
			private function onClose():void
			{
			}
		]]>
	</mx:Script>
	<mx:VBox height="100%" width="100%" verticalAlign="middle" horizontalAlign="center">
		<mx:Button label="Make Demo Document" click="controller.makeDocument()" enabled="true"/>
		<mx:Button label="Make Layer" click="controller.makeLayer()" enabled="{model.demoDocumentOpen}"/>
		<mx:Button label="Move Layer" click="controller.moveLayer()" enabled="{model.demoDocumentOpen}"/>
		<mx:Button label="Merge Layers" click="controller.mergeLayers()" enabled="{model.enableMerge}"/>
		<mx:Button label="Delete Layer" click="controller.deleteLayer()" enabled="{model.demoDocumentOpen}"/>
	</mx:VBox>
</csxs:CSXSWindowedApplication>


The makeDocument functions in the application specific code create an example document in the host application. In Illustrator and InDesign, the functions create page item and assign it to a layer; in Illustrator and Photoshop, the functions add an example layer set/layer group. The application-specific code in these modules might be a useful reference for basic document/layer construction techniques.

AppController

The AppController class is very similar to the corresponding class used in previous posts. AppController includes a set of generic functions that pass parameters off to the application-specific functions for the host application. I’ve used the String type for the parameters—my thinking was that this would make it easy for you to drive the functions from user interface elements.

package layers
{
	import flash.events.Event;
	import flash.filesystem.File;
	import flash.net.FileReference;
	
	public class AppController
	{
		import layers.*;
		private static var instance:AppController;
		private var model:AppModel = AppModel.getInstance();
		
		public static function getInstance() : AppController 
		{
			if (instance == null)
			{
				instance = new AppController();
			}
			return instance;
		}
		public function makeDocument():void{
			switch(model.hostName){
				case "indesign":
					layersInDesign.makeDocument();
					break;
				case "illustrator":
					layersIllustrator.makeDocument();
					break;
				case "photoshop":
					layersPhotoshop.makeDocument();
					break;
			}
		}
		public function makeLayer():void{
			switch(model.hostName){
				case "indesign":
					layersInDesign.makeLayer();
					break;
				case "illustrator":
					layersIllustrator.makeLayer();
					break;
				case "photoshop":
					layersPhotoshop.makeLayer();
					break;
			}												
		}
		public function moveLayer():void{
			switch(model.hostName){
				case "indesign":
					layersInDesign.moveLayer("Layer A", "Layer E", "Before");
					break;
				case "illustrator":
					layersIllustrator.moveLayer("Layer A", "Layer E", "Before");
					break;
				case "photoshop":
					layersPhotoshop.moveLayer("Layer A", "Layer E", "Before");
					break;
			}									
		}
		public function mergeLayers():void{
			switch(model.hostName){
				case "indesign":
					layersInDesign.mergeLayers("Layer B", "Layer D");
					break;
				case "illustrator":
					//No merge layers in Illustrator.
					break;
				case "photoshop":
					layersPhotoshop.mergeLayers("Layer B", "Layer D");
					break;
			}									
		}
		public function deleteLayer():void{
			switch(model.hostName){
				case "indesign":
					layersInDesign.deleteLayer("Layer C");
					break;
				case "illustrator":
					layersIllustrator.deleteLayer("Layer C");
					break;
				case "photoshop":
					layersPhotoshop.deleteLayer("Layer C");
					break;
			}												
		}
		public function handleEvent():void
		{
			switch(model.hostName){
				case "indesign":
					layersInDesign.handleEvent();
					break;
				case "illustrator":
					layersIllustrator.handleEvent();
					break;
				case "photoshop":
					layersPhotoshop.handleEvent();
					break;
			}
		}
	}
}


AppModel

The AppModel class is roughly the same as I’ve presented before—it’s mostly made up of our standard routine for determining the host application of the panel. In this one, I’ve defined a few global variables for the other modules to call on.

public var hostName:String = "";
[Bindable] public var demoDocumentOpen:Boolean = false;
[Bindable] public var enableMerge:Boolean = false;


DocumentWatcher

The DocumentWatcher class is the same as we’ve used in previous posts; see “Watching the Detections” for a detailed discussion of this class. In this example, the application-specific event handler function (handleEvent) looks for a particular bit of data in the active document—if it finds it, it enables some of the buttons in the user interface; if it’s not found, then the function disables the buttons.

Application-Specific Details

All three applications have a move method for layer objects, and all three versions of this method have two parameters: a reference layer and an enumeration. The enumeration specifies where the layer you’re moving should go, relative to the reference layer. The order of the parameters and the type of the enumeration differ among the applications—InDesign wants the location parameter first; Illustrator and Photoshop expect it as the second parameter.

I didn’t want to have to save and name the document, but I needed to mark the document in some way that wasn’t visible. In Illustrator, I found that adding a tag to the page item would provide a way to identify the document; in InDesign, I used a custom label; in Photoshop, I used the document.info object. I could have used XMP metadata to do the same thing, but that would have added unneeded complexity to the project.

Illustrator

package layers
{
	import com.adobe.csawlib.illustrator.Illustrator;
	import com.adobe.illustrator.*;
	
	import flash.desktop.DockIcon;
	
	import layers.AppModel;
	
	public class layersIllustrator
	{
		private static var app:Application = Illustrator.app;
		private static var model:AppModel = AppModel.getInstance();
		private static var instance:layersIllustrator;
		public static function getInstance():layersIllustrator{
			if ( instance == null )
			{
				instance = new layersIllustrator;
			}
			return instance;			
		}
		//makes an example document with five layers.
		public static function makeDocument():void{
			var document:Document = app.documents.add();
			//Remove default layers, if any.
			for(var counter:int = 0; counter < document.layers.length -1; counter++){
				document.layers.index(0).remove();
			}
			var layer:Layer = document.layers.index(0);
			//Rename the remaining default layer (in some cases, Illustrator will not redraw the layer
			//name until you do something (like click layer name). In those cases, the layer
			//name will appear to remain "Layer 1".
			layer.name = "Layer A";
			var subLayer:Layer = layer.layers.add();
			subLayer.name = "Sub-Layer A";
			subLayer = layer.layers.add();
			subLayer.name = "Sub-Layer B";
			//Create and name new layers.
			layer = document.layers.add();
			layer.name = "Layer B";
			layer = document.layers.add();
			layer.name = "Layer C";
			layer = document.layers.add();
			layer.name = "Layer D";
			layer = document.layers.add();
			layer.name = "Layer E";
			//Make "Layer E" the active layer.
			document.activeLayer = layer;
			//Create a rectangle on "Layer E" (if you’re debugging, you’ll have to expand
			//the layer to see the path item in the Layers panel).
			var rectangle:PathItem = document.pathItems.rectangle(144, 72, 72, 72);
			//Move the rectangle to "Layer A".
			sendToLayer("Layer A", rectangle);
			//Add a tag to the rectangle--we’ll use it to identify the example document.
			var tag:Tag = rectangle.tags.add();
			tag.name = "demoDocument";
		}
		public static function makeLayer():Layer{
			if(app.documents.length > 0){
				var document:Document = app.documents.index(0);
				var layer:Layer = document.layers.add();
			}
			return layer;
		}
		public static function deleteLayer(layerName:String):void{
			var layer:Layer;
			layer = getLayerReference(layerName);
			if(layer != null){
				layer.remove();
			}
		}
		public static function getLayerReference(layerName:String):Layer{
			var layer:Layer;
			if(app.documents.length > 0){
				var document:Document = app.documents.index(0);
				try{
					layer = document.layers.getByName(layerName);
				}
				catch(error:Error){}
			}
			return layer;
		}
		public static function moveLayer(layerName:String, referenceLayerName:String, location:String):void{
			var layer:Layer = getLayerReference(layerName);
			if(layer != null){
				var referenceLayer:Layer = getLayerReference(referenceLayerName);
				if(referenceLayer != null){
					layer.move(referenceLayer, getLocationOptions(location));
				}
			}
		}
		public static function getLocationOptions(location:String):ElementPlacement{
			var result:ElementPlacement;
			switch(location){
				case "Before":
					result = ElementPlacement.PLACEBEFORE;
					break;
				case "After":
					result = ElementPlacement.PLACEAFTER;
					break;
			}
			return result;
		}
		//Sends a page item to the specified layer.
		public static function sendToLayer(layerName:String, pageItem:Object):void{
			var layer:Layer = getLayerReference(layerName);
			if(layer != null){
				pageItem.move(layer, ElementPlacement.INSIDE);
			}
		}
		public static function handleEvent():void 
		{
			var result:Boolean = false;
			if(app.documents.length > 0)
			{
				//Put a test here to make sure it’s the demo document.
				var tag:Tag = app.documents.index(0).tags.getByName("demoDocument");
				if(tag.name != null){
					model.demoDocumentOpen = true;
				}
			}
			else{
				model.demoDocumentOpen = false;
			}
		}
	}
}


In the makeDocument function, I’ve shown how to create a layer set. Layers inside the layer set behave in the same fashion as normal layers, and can be manipulated with the other functions in this class.

Trying to get a nonexistent layer in Illustrator produces an error, so I’ve surrounded the attempt with a try...catch statement in the getLayerReference function. Functions that attempt to use the return value from getLayerReference should test for null before attempting to do anything with it.

The Merge Layers button won’t work in Illustrator, because merging layers doesn’t seem to be supported by Illustrator scripting.

InDesign

package layers
{
	import com.adobe.csawlib.indesign.InDesign;
	import com.adobe.indesign.*;
	
	import layers.AppModel;
	
	public class layersInDesign
	{
		private static var app:Application = InDesign.app;
		private static var model:AppModel = AppModel.getInstance();
		private static var instance:layersInDesign;
		public static function getInstance():layersInDesign{
			if ( instance == null )
			{
				instance = new layersInDesign;
			}
			return instance;			
		}
		//makes an example document with five layers.
		public static function makeDocument():void{
			var document:com.adobe.indesign.Document = app.documents.add();
			document.viewPreferences.horizontalMeasurementUnits = MeasurementUnits.POINTS;
			document.viewPreferences.verticalMeasurementUnits = MeasurementUnits.POINTS;
			//Remove default layers, if any.
			for(var counter:int = 0; counter < document.layers.length -1; counter++){
				document.layers.item(0).remove();
			}
			//Rename the remaining default layer.
			document.layers.item(0).name = "Layer A";
			//Create and name new layers.
			var layer:Layer = document.layers.add();
			layer.name = "Layer B";
			layer = document.layers.add();
			layer.name = "Layer C";
			layer = document.layers.add();
			layer.name = "Layer D";
			layer = document.layers.add();
			layer.name = "Layer E";
			//Create a rectangle and assign it to "Layer E".
			var rectangle:Rectangle = document.pages.item(0).rectangles.add(layer);
			//Change the size of the rectangle.
			rectangle.geometricBounds = new Array(72, 72, 144, 144);
			//Move the rectangle to "Layer A".
			sendToLayer("Layer A", rectangle);
			document.insertLabel("demoDocument","true");
		}
		public static function makeLayer():Layer{
			if(!app.modalState){
				if(app.documents.length > 0){
					var document:com.adobe.indesign.Document = app.documents.item(0);
					var layer:Layer = document.layers.add();
				}
			}
			return layer;
		}
		public static function deleteLayer(layerName:String):void{
			var layer:Layer = getLayerReference(layerName);
			if(layer.isValid){
				layer.remove();
			}
		}
		public static function getLayerReference(layerName:String):Layer{
			var layer:Layer;
			if(!app.modalState){
				if(app.documents.length > 0){
					var document:com.adobe.indesign.Document = app.documents.item(0);
					layer = document.layers.item(layerName);
				}
			}
			return layer;
		}
		public static function moveLayer(layerName:String, referenceLayerName:String, location:String):void{
			var layer:Layer = getLayerReference(layerName);
			if(layer.isValid){
				var referenceLayer:Layer = getLayerReference(referenceLayerName);
				if(referenceLayer.isValid){
					layer.move(getLocationOptions(location), referenceLayer);
				}
			}
		}
		public static function mergeLayers(layerName:String, targetLayerName:String):Layer{
			var layer:Layer;
			var document:Document = app.documents.item(0);
			if(document.isValid){
				layer = document.layers.itemByName(layerName);
				var targetLayer:Layer = document.layers.itemByName(targetLayerName);
				if((layer.isValid)&&(targetLayer.isValid)){
					layer = layer.merge(new Array(targetLayer));
				}
			}
			return layer;
		}
		public static function getLocationOptions(location:String):LocationOptions{
			var result:LocationOptions;
			switch(location){
				case "Before":
					result = LocationOptions.BEFORE;
					break;
				case "After":
					result = LocationOptions.AFTER;
					break;
			}
			return result;
		}
		//Sends a page item to the specified layer.
		public static function sendToLayer(layerName:String, pageItem:Object):void{
			var layer:Layer = getLayerReference(layerName);
			if(layer.isValid){
				pageItem.itemLayer = layer;
			}
		}
		public static function handleEvent():void 
		{
			if(!app.modalState){
				var result:Boolean = false;
				if(app.documents.length > 0)
				{
					//A document is open. Is it the demo document we created?
					if(app.documents.item(0).extractLabel("demoDocument") != ""){
						model.demoDocumentOpen = true;
						model.enableMerge = true;
					}
				}
				else{
					model.demoDocumentOpen = false;
					model.enableMerge = false;
				}
			}
		}
	}
}


When you try to get a nonexistent layer in InDesign, the application will happily give you a Layer object (rather than an error and/or null, as in the case of Illustrator and Photoshop). How can you tell you’ve got a real layer? Use the isValid property, as I do in the functions that use the return value of the getLayerReference function.

Photoshop

package layers
{
	import com.adobe.csawlib.photoshop.Photoshop;
	import com.adobe.photoshop.*;
	
	import layers.AppModel;
	
	public class layersPhotoshop
	{
		private static var app:Application = Photoshop.app;
		private static var model:AppModel = AppModel.getInstance();
		private static var instance:layersPhotoshop;
		public static function getInstance():layersPhotoshop{
			if ( instance == null )
			{
				instance = new layersPhotoshop;
			}
			return instance;			
		}
		//makes an example document with five layers.
		public static function makeDocument():void{
			var document:Document = app.documents.add();
			//Remove default layers, if any.
			for(var counter:int = 0; counter < document.artLayers.length -1; counter++){
				document.artLayers.index(0).remove();
			}
			//Create a layer set.
			var layerSet:LayerSet = document.layerSets.add();
			layerSet.name = "Layer Set A";
			var subLayer:Layer = layerSet.artLayers.add();
			subLayer.name = "Sub-Layer A";
			subLayer = layerSet.artLayers.add();
			subLayer.name = "Sub-Layer B";
			//Rename the default layer.
			var layer:ArtLayer = document.artLayers.index(0);
			//Rename the remaining default layer.
			layer.name = "Layer A";
			//Create and name new layers.
			layer = document.artLayers.add();
			layer.name = "Layer B";
			layer = document.artLayers.add();
			layer.name = "Layer C";
			layer = document.artLayers.add();
			layer.name = "Layer D";
			layer = document.artLayers.add();
			layer.name = "Layer E";
			//Make "Layer E" the active layer.
			document.activeLayer = layer;
			//Add a value to the document info so that we can identify the document later.
			document.info.caption = "demoDocument";
			//Update the buttons in the user interface.
			handleEvent();
		}
		public static function makeLayer():Layer{
			if(app.documents.length > 0){
				var document:Document = app.documents.index(0);
				var layer:ArtLayer = document.artLayers.add();
			}
			return layer;
		}
		public static function deleteLayer(layerName:String):void{
			var layer:Layer;
			layer = getLayerReference(layerName);
			if(layer != null){
				layer.remove();
			}
		}
		public static function getLayerReference(layerName:String):Layer{
			var layer:Layer;
			if(app.documents.length > 0){
				var document:Document = app.documents.index(0);
				try{
					layer = document.layers.getByName(layerName);
				}
				catch(error:Error){}
			}
			return layer;
		}
		public static function moveLayer(layerName:String, referenceLayerName:String, location:String):void{
			var layer:Layer = getLayerReference(layerName);
			if(layer != null){
				var referenceLayer:Layer = getLayerReference(referenceLayerName);
				if(referenceLayer != null){
					layer.move(referenceLayer, getLocationOptions(location));
				}
			}
		}
		//Both ArtLayers and LayerSets can be merged, so use Object instead of Layer.
		public static function mergeLayers(layerName:String, targetLayerName:String):Object{
			var layer:Object = getLayerReference(layerName);
			if(layer != null){
				var targetLayer:Object = getLayerReference(targetLayerName);
				if(targetLayer != null){
					//Photoshop merges the layer with the layer below it,
					//so move the target layer above the layer, then merge.
					targetLayer.move(layer, getLocationOptions("Before"));
					targetLayer.merge();
				}
			}
			return layer;
		}
		public static function getLocationOptions(location:String):ElementPlacement{
			var result:ElementPlacement;
			switch(location){
				case "Before":
					result = ElementPlacement.PLACEBEFORE;
					break;
				case "After":
					result = ElementPlacement.PLACEAFTER;
					break;
			}
			return result;
		}
		public static function handleEvent():void 
		{
			var result:Boolean = false;
			if(app.documents.length > 0)
			{
				//Test to make sure it’s the demo document.
				if(app.activeDocument.info.caption == "demoDocument"){
					model.demoDocumentOpen = true;
					model.enableMerge = true;
				}
			}
			else{
				model.demoDocumentOpen = false;
				model.enableMerge = false;
			}
		}
	}
}


In the makeDocument function, I’ve added a layer group (LayerSet). The layers inside a layer group behave just like any other layers in Photoshop, and can be manipulated with the functions in this class.

Photoshop, like Illustrator, will throw an error when you try to get a nonexistent layer, so test for null anytime you get the return value of the getLayerReference function.

Testing the Extension

I’m hoping this is fairly straightforward. Click the Make Sample Document button. The host application will create a new document with a set of layers.

Click the Move Layer button. The host application should move the layer named “Layer A” above “Layer E.”

Click the Delete Layer button. The host application should remove the layer named “Layer C.”

Click the Merge Layers button (in InDesign and Photoshop). The host application should merge “Layer D” into “Layer B.”

Click the Make Layer button. The host application should create a new layer. There’s really not much to this one, but I included it to show that the new layer will appear at the top of the Layers panel and given a default name.

Layers are Cake

While managing layers isn’t the most glamorous thing you can do in a Creative Suite Extension, I think it’s something that we’ll all have to do at some point. My hope is that these posts on basic techniques for scripting Creative Suite applications save someone, somewhere, some time.

Given all of the different layer types and behaviors in Photoshop, I’m inclined to do another post on that topic. Sound interesting? If you have a comment, or if you have an idea for a future post, please let me know.

One Response to Working with Layers

  1. Hello Olav,
    thanks for the post (neat idea to use info.caption to identify the document!). I’m appreciating the alternation between basic and more advanced topics. The latter help learning, the former may be really time savers.
    Kind regards

    Davide