You are here: osflash » hamtasc

HAMTASC - Hacking Mtasc

Nicolas Cannasse created and maintains the great open source actionscript compiler mtasc, which leaves little to be desired. But besides beeing able to compile Actionscript with it, his generous present to the open source flash community gives you a complete swf creation framework at your command. All you need to know is a little OCaml. Learning a new language from time to time makes you an even better programmer. So what you are waiting for, start hacking mtasc! And don’t forget to support Nicolas for his great work. You can find a little donate button at the bottom of the mtasc website.

Downloads / Uploads

Aral Balkan generously hosts a subversion repository for our experiments.

In order to download the files, you’ll need a subversion client. I’m using TortoiseSVN on windows, which seems to be working very well.

You can always find the latest windows build of mtasc at http://svn1.cvsdude.com/osflash/mtaschacks/bokel/bin/

The root directory of the hamtasc source can be found at http://svn1.cvsdude.com/osflash/mtaschacks/ and my stuff in the subdirectory bokel. The repository is open for reading. If you also need write access to the repository, ask Aral for a password.

Prerequisites

Ralf Bokelberg:I’m using the cygwin variant to compile mtasc. It is completely free. Here is how to setup:

  1. Download and install Cygwin including gcc, make, mingw and zlib
  2. Download and install ocaml (mingw version). Don’t download MinGW, since you don’t need it !!
  3. Because ocaml doesn’t correctly set the Sys.os_type, we have to tweak the main install.ml from the mtasc website. Bind a new variable os_type to “Cygwin” and replace all occurences of Sys.os_type by os_type. There is another install.ml inside the extlib-dev directory which needs the same procedure.
(* ----- BEGIN CONFIGURATION ---- *)

(* set the path to the libz :*)
let zlib = "C:\\Programme\\cygwin\\lib\\mingw\\libz.a."


let bytecode = false
let native = true

(* set the os_type by hand *)
(*let os_type = "Win32" *)
let os_type = "Cygwin" 

(* ------ END CONFIGURATION ----- *)

let obj_ext = match os_type with "Win32" -> ".obj" | _ -> ".o"
let exe_ext = match os_type with "Win32" | "Cygwin" -> ".exe" | _ -> ""

Now you should be able to compile mtasc using the command “ocaml install.ml” from a Cygwin shell. If you set the path to Cygwin correctly, you can also use a cmd window, otherwise install.ml will complain about a missing as (the assembler to create the native code).

For those unfamiliar with cygwin, you must install oCaml from ‘inside’ cygwin. OCaml will become available from the command line. You MUST NOT install the oCaml version from http://caml.inria.fr/download.en.html. This version will not compile correctly under cygwin. Additionally, if you have downloaded and installed this version, you must uninstall and manually unset the OCAMLLIB environment variable for the cygwin version to function correctly. To unset the variable go to Control Panel→System→Advanced→Environment Variables. Select the OCAMLLIB variable under System Variables and press Delete.

Please note that the cygwin version relies on the cygwin install. So for this reason if you want to run your mtasc.exe from within ant on a normal command prompt, or perhaps via eclipse, you will be required to add cygwin’s path to your PATH environment variable. For example, you should add C:\cygwin\bin;C:\cygwin\lib in order for programs outside of cygwin to get access to your compiled binary. To set the variable go to Control Panel→System→Advanced→Environment Variables. Select the PATH variable under System Variables and press Edit. Add the paths, using semicolons to separate them.

Adding ASSERT

Ralf Bokelberg: Adding ASSERT is simple. Open ocaml/mtasc/genSwf.ml and duplicate the block regarding TRACE. Replace TRACE by ASSERT and ftrace by fassert. Then search for other occurences of ftrace, duplicate them too while also replacing ftrace with fassert. Finally add ASSERT as a method to std/TopLevel.as. That’s it. Now you are able to add calls to ASSERT all over your code and switch them on/off globally by passing the appropriate -assert option to mtasc.

In the following you can see a small example of ASSERT. In principal it works just like TRACE. Compile it using mtasc -assert MainClass.assertImpl ...

class MainClass
{
	public static function main(){
		trace("main");
		ASSERT(arguments.length > 0 && arguments[0] instanceof MovieClip);
	}
	
	private static function assertImpl( expectedToBeTrue:Boolean){
		trace("assertImpl " + arguments);
	}
}

Adding test of Void parameter

Ralf Bokelberg: Some people like using Void to show, that a method doesn’t take parameters, eg. function test(Void) should not accept any parameters. The idea is to automatically assume type Void, when an argument named Void is used. Extending parse_args (in parser.ml) like that seems to do the trick:

   and parse_args = parser
  | [< '(Const (Ident "Void"),_); t = parse_type_option; '(PClose,p) >] ->
      if  t <> None then error CannotTypeVoid p;
      ("Void" , Some ([],"Void")) :: [], p
  | [< '(Const (Ident name),_); t = parse_type_option; al , p = parse_args >] -> (name , t) :: al , p
  | [< '(Sep,_); al= parse_args >] -> al
  | [< '(PClose,p) >] -> [] , p

Additionally you have to extend error_msg with the new constructor named CannotTypeVoid.

class MainClass {
    public static function main(){
         test( 1, 2);     
    }
 
    private static function test(Void){
         trace("test: We don't want parameters: " + (arguments.length == 0));
    }
}

Adding a parameter from the build to main

Ralf Bokelberg: Passing a parameter from the build parameters to the swf is easy. We add another option fparam by copying the lines where ftrace is set to ref None and is added to Plugin at the end of genSwf.ml. Make sure to replace ftrace by fparam in both cases. Now we can pass the parameter to the generator.

(* at the top of genSwf.ml *)
let ftrace = ref None
let fparam = ref None
 
(* at the bottom of genSwf.ml *)
("-trace",Arg.String (fun t -> ftrace := Some t),"<function> : specify a TRACE function");
("-param",Arg.String (fun t -> fparam := Some t),"<function> : specify a parameter string for main.");

All we have to do now is to slightly tweak the code generated to call main. Search for the comment (*// (main class).main(this); *) in generate. The old code looks like this:

push ctx [VThis];
write ctx AEval;
push ctx [VInt 1];

we replace it by this code, which adds our param string as additional parameter to main.

push ctx [VStr (match !fparam with None -> "" | Some f -> f)];		
push ctx [VThis];
write ctx AEval;
push ctx [VInt 2];

Now calling main should result in a call with two parameters: the timeline and the string given in param or the empty string.

Preprocessing Actionscript

Ralf Bokelberg:To preprocess our Actionscript files we have to replace the content of the channel read by the lexer by the preprocessed content. The classes are read in load_file (in typer.ml). The following change to load_file adds a preprocessing stage if a parameter named preprocessor is set.

(* all we need to change is the first local function in load_file and to add the additional option to mtasc *)
let rec loop = function
		| [] -> raise (File_not_found file)
		| path :: paths ->
			try
				let file = path ^ file in
				let ch = open_in file in
				match !preprocessor with 
				| None -> file, ch (* the proprocessor option is not set, return the normal channel
				| Some f -> 
                                        (* the preprocessor option is set *)
                                        (* close the old in channel *)
					close_in ch;
                                        (* create a temporary filename *)
					let tmp_file = Filename.temp_file "" ".as" in
                                        (* call the preprocessor with the original filename and the temporary filename *)
					let sys_result = Sys.command( f ^ " '" ^ file ^ "' '" ^ tmp_file ^ "'") in 
                                        (* return the old filename but open the temporary filename as our input channel *)
					file, open_in tmp_file 
			with
				_ -> loop paths
	in

And my ant commandline looks like this: <arg value=”-preprocessor”/><arg value=”C:\Programme\cygwin\bin\cpp -E -P -w”/> Attention: Preprocessing significantly slows down the compilation.

-rb_entry takes entry point parameter

If you want to specify the entry point of your application, -rb_entry <class.method> is for you. It enables you to use any public static method as your entry point.

Setting the ScriptLimits tag

Ralf Bokelberg: The F7 player allows for adjusting the maximum recursion depth (default 256) and the seconds until a script timeout is raised (default 15). If we want to set these values from mtasc, we have to add another parameter scriptlimit. This time we copy header (in genSwf.ml) and the function make_header and tweak them to our needs. Also we add a new type script_limits and a new tag constructor TScriptLimits to swf.ml. Finally we have to add parser routines to read and write TScriptLimits tags (in swfParser.ml).

(* changes to genSwf.ml *)
 
(* initialize the new option at the top *)
let script_limits = ref None
 
(* add the new option to the argument table at the bottom *)
("-scriptlimits",Arg.String (fun s -> script_limits := Some (make_script_limits s)),"<scriptlimits> : specify scriptlimits format 'max_recursion_depth:script_timeout_seconds'");
 
(* duplicate make_header and change it to make_script_limits *)
let make_script_limits s =
	let sl = String.nsplit s ":" in
	try
		let make mr st =
			let mr = int_of_string mr in
			let st = int_of_string st in
			{
				sl_max_recursion_depth = mr;
				sl_script_timeout_seconds = st;
			}
		in
		match sl with
		| [mr;st] ->
			make mr st
		| _ ->
			raise Exit
	with
		_ -> raise (Arg.Bad "Invalid scriptlimits format")
 
(* add some code to generate which inserts the additional tag *)
        (* add the code after this line *)
	let header = (if !flash6 then { header with h_version = 6 } else header) in
	let data = ( match !flash6, !script_limits with 
		| false , Some sl ->
			[tag (TScriptLimits {sl_max_recursion_depth = sl.sl_max_recursion_depth; sl_script_timeout_seconds = sl.sl_script_timeout_seconds} ) ] @ data
		| _ , _ -> 
			data) 
	in 
 
 

Compile one class only

Ralf Bokelberg: Daniel Fischer asked for the possibility to compile one class only. To achieve that we add another option single_class which allows for specifiying the class to output. All we have to do then is to compare this class to the classname of the classes to be generated

(* our new option needs to be initialized at the top of genSWF.ml *)
let single_class = ref None

(* the new option needs to be added near the bottom of genSWF.ml too *)
("-single_class",Arg.String (fun f -> single_class := Some f),": <class> : output single class only");

(* search for "if not (Class.intrinsic clctx) && not (Hashtbl.mem excludes (s_type_path (Class.path clctx)))" and replace it by *) 
 if not (Class.intrinsic clctx) 
 && not (Hashtbl.mem excludes (s_type_path (Class.path clctx))) 
 && ( !single_class = None || Some (s_type_path (Class.path clctx)) = !single_class) then begin

Now, if you specify -single_class de.bokelberg.TestClass as parameter to mtasc only de.bokelberg.TestClass is created in the output swf.

Checking constructor name

Ralf Bokelberg: Every so often my code breaks because my constructor is named different from my class. This happens whenever i rename a class but forget to rename the constructor. These kind of bug is very hard to find.

Today i had the idea to let mtasc do it, and it actually works quite good. All i do is to compare every method starting with an uppercase letter to the name of the current class. Whenever they are different, mtasc spits out a warning.

Attention: This method assumes, that the constructor and the class begin with an uppercase letter and that other methods begin with a lowercase letter.

(* In typer.ml you can find the function type_function. Insert the code at the beginning of the first "Some e" match 
if not lambda then verbose_msg ("Typing " ^ s_type_path clctx.path ^ "." ^ f.fname);
if( String.length f.fname > 0 && Char.uppercase f.fname.[0] = f.fname.[0] && f.fname.[0] <> '_' && clctx.name <> f.fname) then (!Parser.warning) "Possibly incorrect constructor name" p;

SWC Support

Ralf Bokelberg: Jonathan Doklovic asked for SWC support in MTasc. He wanted to add swc files just like directories to the classpath and let MTasc use the intrinisc class files from inside the swc.

<example snippet>
mtasc -cp "(path to mm classes)" -mx -swc "(path to swc files)" -swf test.swf Test.as
</example snippet>

Instead of building this functionality into Mtasc, we can do it using ant. I wrote a little ant script, which unpacks a file test.swc to the directory swc/test/. If you add this directory to the classpath of MTasc it should work as expected.

<macrodef name="unpackSwc">
	<attribute name="swcName" default=""/>
	<attribute name="targetDir" default="swc"/>	
	
	<sequential>
		<unzip dest="@{targetDir}/@{swcName}/">
			<patternset>
			        <include name="**/*.asi"/>
			</patternset>	
			<fileset file="@{swcName}.swc"/>
		</unzip>
		<copy todir="@{targetDir}/@{swcName}/">
		    <fileset dir="@{targetDir}/@{swcName}" includes="**/*.asi"/>
			<mapper type="unpackage" from="*.asi" to="*.as"/>
	  	</copy>
		<delete>
			<fileset dir="@{targetDir}/@{swcName}" excludes="**/*.as"/>
		</delete>	
	</sequential>	
</macrodef>

<target name="unpack-myLib.swc">
	<unpackSwc swcName="myLib"/> 
</target>

Checking for unused private members

Ralf Bokelberg: To check for private members, which are never used, we have to do two things. First we have to count accesses to the member and second we have to add a finalizer, which steps through all members and checks if a member has been accessed.

(* all changes in typer.ml *)
(* add a field to check access to the member records *)
and class_field = {
	f_name : string;		(* the name of the field *)
	f_type : type_decl;		(* the type of the field *)
	f_static : static_flag; (* is it static? *)
	f_public : public_flag; (* is it public? *) 
	mutable f_used : bool;  (* is it used (privates only) <------ added *)
	f_pos : pos;			(* position in source *)
}
 
 
(* add check, if member is accessed *)
let rec resolve t fname =
	match t with
	| Void
	| Dyn
	| Function _ -> None
	| Package p ->
		Some {
			f_name = fname;
			f_type = Package (p @ [fname]);
			f_static = IsMember;
			f_public = IsPublic;
			f_pos = null_pos;
			f_used = false;
		}
	| Static c -> 
		(try Some (Hashtbl.find c.statics fname) with Not_found -> if c.super == c then None else resolve (Static c.super) fname)
	| Class c -> 
		try 
			let result = (Hashtbl.find c.fields fname) in 
			result.f_used <- true;       (* we have been accessed, set field to true *)
			Some (Hashtbl.find c.fields fname)
			
		with 
			Not_found -> 
				if c.super == c then
					None 
				else 
					resolve (Class c.super) fname
 
 
(* add the finalizer, executed after the class is typed *)
let load_class ctx path p =
	try
		Hashtbl.find ctx.classes path
	with
		Not_found ->
			if String.lowercase (snd path) = "con" then error (Custom "CON is a special file under Windows and shouldn't be used as class name") p;
			let file_name = (match fst path with
				 | [] -> snd path ^ ".as"
				 | _ -> String.concat "/" (fst path) ^ "/" ^ snd path ^ ".as")
			in
			try
				match type_file ctx path file_name (load_file ctx file_name) p with
				| None -> error (Custom "Missing class definition") { pfile = file_name; pmin = 0; pmax = 0 }
				| Some c -> 
                                        (* this is the point where out new finalizer is added *)
					add_finalizer ctx (fun () ->
						let name = c.name in
						Hashtbl.iter (fun _ f -> 
							if f.f_public = IsPrivate && not f.f_used then prerr_endline ("Warning: private member not used: " ^ f.f_name)
						) c.fields;
						
						Hashtbl.iter (fun _ f -> 
							if f.f_public = IsPrivate && not f.f_used && name <> "TopLevel" then prerr_endline ("Warning: private member not used: " ^ name ^ ":" ^ f.f_name)
						) c.statics;
						
					);	
					c
			with
				File_not_found _ -> error (Class_not_found path) p
 
 

Checking for default constructors called with parameters

Ralf Bokelberg: If you don’t specify a constructor, the defualt constructor is used. Now if you pass parameters to the constructor, it could be an error, because the defualt constructor doesn’t know what to do with the parameters. I added a new option -rb_check_default_constructor_doesnt_take_param which outputs a warning in this case.

Suppressing "Import not used" Warnings

Ralf Bokelberg: Sometimes you don’t like these warnings, because other, more important warnings may be hidden by them. Now you are able to suppress them by providing -rb_suppress_import_warnings to mtasc.

The switch name seems to be outdated. The current hamtasc (and mtasc ) version only has a -wimp (warn imports) switch which defaults to false e.g. you have to call mtasc with -wimp to enable warnings.

However, these warnings are useful in that you can use them to remove the imports from your class files. See mtasc_remove_imports for more information.

Getting excludes from swf

Ralf Bokelberg: Instead of manually providing a list of files to exclude you can simply use a swf to do the very same. The new option -rb_exclude_swf <swffile> does just that. It retrieves all classes from swffile and uses them as exclude list.

State

The hacked mtasc seems to be reliable and stable. I’m using it everyday in my production environment. Especially the additional check options and the possibility to pass a param to main, make it very valuable to me.

Features

Additionally to the great features of the original mtasc, the hacked mtasc provides you with the following checks

  • check for missing or additional commas in arrays
  • check for missing or additional commas in objects
  • check for multiple var keywords
  • check for missing var declarations, eg var without the following stuff
  • check for multiple static keywords
  • check for multiple private/public keywords
  • wildcards in exclude files are handled like rb_exclude_package (see below)
  • import not used - warnings also contain the name of the import
  • private members of surrounding class can be accessed in anonymous function
  • __LINE__, __CLASS__, __METHOD__, __FILE__ are replaced by their actual value (inside strings only)

Options

Here is the list of additional options. They all start with “rb_” to avoid conflicts with future changes of the original mtasc.

-rb_assert <function>

specify an ASSERT function (like in trace)

-rb_auto_trace <function>

specify an auto trace function (experimental, the trace function is inserted at the beginning of every function. This function needs to have the following signature: function (classname:String, methodname:String, filename:String, lineNumber:Number, scope:Object, arguments:ArgumentsObject)

-rb_auto_trace_pop <function>

similar to -rb_auto_trace, but this function is inserted at every exit point of a function. If you use it together with rb_auto_trace, you can easily create a stacktrace. The supplied function has the following signature: function (lineNumber:Number)

-rb_check_constructor

turn on checking of constructor. This function warns you, if it finds a function starting with uppercase character, which has a different name than the constructor.

-rb_check_default_constructor_doesnt_take_param

check if default constructor is called with params. If you don’t specify a constructor, flash automatically uses the default constructor. Now if you pass parameters to the constructor, it is very likely, that you forgot to create a custom constructor, because the default constructor doen’t take parameters.

-rb_check_function_call_missing

turn on checking of missing (). If you have sequences of function calls, it may happen, that you forget the (), eg. MyObject.getSubObject.callMethod(). This will give you a warning, because then function getSubObject is used without ()

-rb_check_local_shadows_member_var

checks, if a local variable shadows a member var inside a function

-rb_check_local_shadows_member_var_in_anonymous_function

checks, if a local variable, which is accessed inside an anonymous function, shadows a member var. This is particular interesting, because MMC fails to compile such code.

-rb_check_unused_privates

checks, if a private member of a class is used inside the class

-rb_check_void_parameter

turn on checking of void parameter. Some people like passing Void as single parameter to functions, which doesn’t take a parameter. This option gives you a warning, if you pass a parameter to such a function. Note: This kind of check only works in the hacked mtasc. Other compilers simply create a Parameter called Void, which may slowdown your program

==== -rb_entry <function>==== specify main entry point. Mtasc automatically searches for the public static main in the current source tree. If you want to specify the main function explicitely, this option is for you. Nice for testing. If you activate verbose mode, the entry point will be printed to your console. Note: If you provide multiple public static main functions, mtasc will complain about it. Therefore you better use another name for yuor other entry points.

-rb_exclude_swf <swffile>

exclude classes included in swf. Instead of providing a list of classes you can provide a swf. All the classes, which are part of this swf, are added to the internal exclude list and thus, excluded. If you activate verbose mode, the excluded classes are printed to the console like this: Excluding mypackage.MyClass

-rb_exclude_package <package>

-rb_exclude_package com.mypackage excludes all classes, which start com.mypackage. Note: This is a very simple string comparison and not a recursive search for all depending classes. If you classes reference classes outside of com.mypackage, this classes still are included.

-rb_intrinsic_out_path <dir>

creates intrinsics of all classes seen during the compile and write them to dir

-rb_intrinsic_suffix <suffix>

change the default .asi suffix to whatever you like

-rb_intrinsic_java_out_path <dir>

creates intrinsic java classes of all classes seen during the compile and write them to dir. This is useful in connection with j2as only

-rb_list_included_classes

lists included classes to stdout

-rb_pass_param <string>

specify a string to be passed to main as second parameter. Additionally to the timeline parameter you can specify a custom string parameter for your main function here. This is very useful, if you want to pass in a path or a version string for example.

-rb_preprocessor <command>

specify the commandline to invoke the preprocessor. this option enables you to include a classic preprocessor like cpp into your workflow. Note: Compiling is much slower this way, because the cpp is called on every file mtasc compiles.

-rb_script_limits <script_limits>

specify script limits ‘max_recursion_depth:script_timeout_seconds’. If you need to have a bigger recursion stack than 255 or you need some more time than 15 seconds, before the timeout message appears, you can set this options of the flash player here.

-rb_single_class <class>

output single class only. This is an experimental feature for people who like to build using make

-rb_suppress_import_warnings

suppress ‘import not used’ warnings

Wishlist

So you don’t want to learn something new or you are afraid of breaking something, but still want this little feature, which would boost yout productivity by 200% ? Tell us about it.

  • provide a strict mode, with output trace anytime an undefined variable was used or an undefined function called.
  • clean up this strange swf/out/header logic
  • inject version string
    • this would be done using version control keyword substitution. E.g. in CVS put a keyword in the code (var s:String = “$Name$”) and then export the project and compile. Exporting does keyword substitution (var s:String = “V10.04.02”). Typically, you would use the current tag to be substituted.
      • interesting technique, but it limits you to strings of the same length
      • How so? CVS has no limits on the length of tag. Remember the keyword substitution is applied to the source before it’s compile. I believe you can do keyword substitution on binary files, but I wouldn’t do it for files that are generated from the source.
      • ah, now i understand, i thought you ment replacing it in the swf. But isn’t exporting a little bit too much effort, just to compile with a new version string? Actually i never used export, probably i’m missing the point.
  • your wish here
  • optimize our compiled swfs (I don’t know if mtasc does these already)
    • At least really small optimizations like remove double “not” on == operator.
  • compute things like (4*5) at compiletime
    • var i:Number = (4*5); ??? variable declaration with initial value, the initial value is evaluated at compile time.
    • var j:Number = i + 5/4*3; This is not evaluated (even if I use ()) [isn’t that because the compiler can’t or isn’t determining that i is also a constant?][`i` is not a constant, so of course that could not be optimized. But 5/4*3 IS a constant and the above expression could be simplified as j = i+3.75;
  • > do other fancy optimization tricks (unrolling loops? I’m not a compiler expert. Maybe look at the gcc optimization options for tips?)
  • support an ‘inline’ keyword on functions that places the content of that function inline when called, instead of a call to that function
  • support #DEFINE:
    • #DEFINE MY_VALUE:Number = 2;
    • #DEFINE MY_VALUE2:Number = 1;
    • var i = MY_VALUE; should be compiled: var i = 2;
    • var j = MY_VALUE | MY_VALUE2; should be compiled: var j = 3;
  • use the header option to modify the size of a SWF without the need of compiling any AS
  • have more syntaxic keywords like in kinetic fusion : const final and abstract.
  • -rb_include_package <package> option to compile only classes started with specified package.
  • use SWF files as a class sources in the -cp argument. HAMTASC shell search SWF for required class and move compiled code from this SWF to the target application.
  • Optionally compile embedded Flasm bytecode instructions. Flash IDE allows these strings, but MTASC returns an error.



Recent changes RSS feed Creative Commons License Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki
About - Site design by Aral Balkan, based on the original design by Nicolas Coevoet. OSFlash © 2005 Aral Balkan.
OSFlash, the OSFlash logo and mascot are trademarks of Ariaware Limited.

You are here: osflash » hamtasc OSFlash footer graphic (graffiti).