Quantcast

Compiling a Scala Snippet at run-time

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Compiling a Scala Snippet at run-time

Colin Howe
I've had a look around and can't find anything... but I'm hoping this is possible: I have a file "some-script.scala" (or even just an arbitrary string of scala code). At run-time I want to do something like:
  val compiledScript = compile("some-script.scala")
  compiledScript.class("ClassInScript").method("run").execute
(I've tried to keep this succinct to get the point across but I realise that a real solution will likely be more verbose) Any suggestions / ideas?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [scala] Compiling a Scala Snippet at run-time

Matt Hildebrand
Colin,

It is possible.  The basic idea is to invoke the Scala compiler at runtime, then use a classloader to load the resulting bytecode into the JVM.

Some code that does this, for your reference, is available here:
In particular, check the preparePage() method in
    serverpages/source/scala/com/mh/serverpages/ServerPageServlet.scala
for an overview of the process, and
    serverpages/source/scala/com/mh/serverpages/scala_/ScalaCompiler.scala
for an example of invoking the compiler.  The Scala REPL source may also be a useful reference, but I'm less familiar with it and cannot point you directly to the relevant parts.

To keep things simple, this code has the compiler read the source from, and write the bytecode to, files on the filesystem.  If I recall correctly, the compiler source also makes available a way to do the whole operation in memory only.

(The code referenced above is my implementation of Scala Server Pages -- like JSPs, but for Scala.  While this project was not particularly useful -- it was mainly just something to build while learning Scala -- it does happen to do what you describe, and so may be a useful reference.  Hope it helps...)

Best,
-Matt



On Fri, Sep 11, 2009 at 5:04 AM, Colin Howe <[hidden email]> wrote:
I've had a look around and can't find anything... but I'm hoping this is possible: I have a file "some-script.scala" (or even just an arbitrary string of scala code). At run-time I want to do something like:
  val compiledScript = compile("some-script.scala")
  compiledScript.class("ClassInScript").method("run").execute
(I've tried to keep this succinct to get the point across but I realise that a real solution will likely be more verbose) Any suggestions / ideas?

View this message in context: Compiling a Scala Snippet at run-time
Sent from the Scala mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [scala] Compiling a Scala Snippet at run-time

Bryan Castillo

Matt Hildebrand wrote
Colin,
It is possible.  The basic idea is to invoke the Scala compiler at runtime,
then use a classloader to load the resulting bytecode into the JVM.

Some code that does this, for your reference, is available here:
    http://github.com/matthild/serverpages/tree/master
In particular, check the preparePage() method in
    serverpages/source/scala/com/mh/serverpages/ServerPageServlet.scala
for an overview of the process, and
    serverpages/source/scala/com/mh/serverpages/scala_/ScalaCompiler.scala
for an example of invoking the compiler.  The Scala REPL source may also be
a useful reference, but I'm less familiar with it and cannot point you
directly to the relevant parts.

To keep things simple, this code has the compiler read the source from, and
write the bytecode to, files on the filesystem.  If I recall correctly, the
compiler source also makes available a way to do the whole operation in
memory only.

(The code referenced above is my implementation of Scala Server Pages --
like JSPs, but for Scala.  While this project was not particularly useful --
it was mainly just something to build while learning Scala -- it does happen
to do what you describe, and so may be a useful reference.  Hope it
helps...)

Best,
-Matt



On Fri, Sep 11, 2009 at 5:04 AM, Colin Howe <colinthehowe@gmail.com> wrote:

> I've had a look around and can't find anything... but I'm hoping this is
> possible: I have a file "some-script.scala" (or even just an arbitrary
> string of scala code). At run-time I want to do something like:
>
>   val compiledScript = compile("some-script.scala")
>   compiledScript.class("ClassInScript").method("run").execute
>
> (I've tried to keep this succinct to get the point across but I realise
> that a real solution will likely be more verbose) Any suggestions / ideas?
> ------------------------------
> View this message in context: Compiling a Scala Snippet at run-time<http://www.nabble.com/Compiling-a-Scala-Snippet-at-run-time-tp25397478p25397478.html>
> Sent from the Scala mailing list archive<http://www.nabble.com/Scala-f14147.html>at Nabble.com.
>

I wanted to try and invoke the scala compiler recently without using the Filesystem.  Looking around the scala-tools source, I noticed that there was an Interpreter class.  I was able to use that to compile snippets to bytecode in memory.  The Interpreter seems to use a virtual filesystem in memory and a special class loader.  I had trouble extracting results or compiled classes from the ClassLoader the Interpreter created though.  I was able to throw in a ThreadLocal hack and a regex to test out this process.  It seems to be working for a prototype I am putting together.  I want to refine the code so that it doesn't seem like such a hack, but I think this does demonstrate that the concept works.


public class ScalaFunctionFactory extends ByteCodeCompilingFunctionFactory {

    private static Pattern findClasses = Pattern.compile("^\\s*class\\s+([a-zA-Z0-9_\\$]+)", Pattern.MULTILINE);
   
    private static ThreadLocal<Class> lastDefinedClass = new ThreadLocal<Class>();
   
    public static void notifyDefinedClass(Class cl) {
        lastDefinedClass.set(cl);
    }
   
    @Override
    protected Class<Function> compile(String source, String sourceType) throws FunctionException {
   
        // Extreme hackery:
        // This is using the scala interpreter to compile scala code.
        // The interpreter doesn't expose the class loader it uses so code is injected into
        // the code to interpret to return the class through a thread local variable.
        // This also requires trying to figure out which class was meant to be defined using
        // a regex.
       
        Matcher matcher = findClasses.matcher(source);
        if (!matcher.find()) {
            throw new FunctionException("Could not find a defined class.");
        }
        String className = matcher.group(1);
       
        Settings settings = new Settings();
        Interpreter interpreter = new Interpreter(settings);
   
        lastDefinedClass.set(null);
        interpreter.interpret(
            source + "\n" +
            ScalaFunctionFactory.class.getName() + ".notifyDefinedClass(classOf[" + className + "])\n"
        );
       
        Class clazz = lastDefinedClass.get();
        if (clazz == null) {
            throw new FunctionException("No class defined in source.");
        }
       
        return clazz;
       
    }

}


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [scala] Compiling a Scala Snippet at run-time

Bill Burdick-2
Here's something that may work for you.


import scala.tools.nsc.{Interpreter,Settings}

var i = new Interpreter(new Settings(str => println(str)))
var res = Array[Any](null)
i.beQuietDuring({
i.bind("result", "Array[Any]", res)
i.interpret("class Fred {def foo = 3}")
i.interpret("result(0) = new Fred")
})

println(res(0).asInstanceOf[{def foo: Int}].foo)


i.beQuietDuring(function) suppresses output like "defined class Fred".
i.bind(name, type, value) sets values in the environment which you can to share objects with the interpreter.
You can share types as well, just interpret an import statement (so you don't need asInstanceOf[] exprs).


Bill


On Fri, Jan 15, 2010 at 3:56 AM, castillo.bryan <[hidden email]> wrote:



Matt Hildebrand wrote:
>
> Colin,
> It is possible.  The basic idea is to invoke the Scala compiler at
> runtime,
> then use a classloader to load the resulting bytecode into the JVM.
>
> Some code that does this, for your reference, is available here:
>     http://github.com/matthild/serverpages/tree/master
> In particular, check the preparePage() method in
>     serverpages/source/scala/com/mh/serverpages/ServerPageServlet.scala
> for an overview of the process, and
>     serverpages/source/scala/com/mh/serverpages/scala_/ScalaCompiler.scala
> for an example of invoking the compiler.  The Scala REPL source may also
> be
> a useful reference, but I'm less familiar with it and cannot point you
> directly to the relevant parts.
>
> To keep things simple, this code has the compiler read the source from,
> and
> write the bytecode to, files on the filesystem.  If I recall correctly,
> the
> compiler source also makes available a way to do the whole operation in
> memory only.
>
> (The code referenced above is my implementation of Scala Server Pages --
> like JSPs, but for Scala.  While this project was not particularly useful
> --
> it was mainly just something to build while learning Scala -- it does
> happen
> to do what you describe, and so may be a useful reference.  Hope it
> helps...)
>
> Best,
> -Matt
>
>
>
> On Fri, Sep 11, 2009 at 5:04 AM, Colin Howe <[hidden email]>
> wrote:
>
>> I've had a look around and can't find anything... but I'm hoping this is
>> possible: I have a file "some-script.scala" (or even just an arbitrary
>> string of scala code). At run-time I want to do something like:
>>
>>   val compiledScript = compile("some-script.scala")
>>   compiledScript.class("ClassInScript").method("run").execute
>>
>> (I've tried to keep this succinct to get the point across but I realise
>> that a real solution will likely be more verbose) Any suggestions /
>> ideas?
>> ------------------------------
>> View this message in context: Compiling a Scala Snippet at
>> run-time<http://www.nabble.com/Compiling-a-Scala-Snippet-at-run-time-tp25397478p25397478.html>
>> Sent from the Scala mailing list
>> archive<http://www.nabble.com/Scala-f14147.html>at Nabble.com.
>>
>
>


I wanted to try and invoke the scala compiler recently without using the
Filesystem.  Looking around the scala-tools source, I noticed that there was
an Interpreter class.  I was able to use that to compile snippets to
bytecode in memory.  The Interpreter seems to use a virtual filesystem in
memory and a special class loader.  I had trouble extracting results or
compiled classes from the ClassLoader the Interpreter created though.  I was
able to throw in a ThreadLocal hack and a regex to test out this process.
It seems to be working for a prototype I am putting together.  I want to
refine the code so that it doesn't seem like such a hack, but I think this
does demonstrate that the concept works.


public class ScalaFunctionFactory extends ByteCodeCompilingFunctionFactory {

   private static Pattern findClasses =
Pattern.compile("^\\s*class\\s+([a-zA-Z0-9_\\$]+)", Pattern.MULTILINE);

   private static ThreadLocal<Class> lastDefinedClass = new
ThreadLocal<Class>();

   public static void notifyDefinedClass(Class cl) {
       lastDefinedClass.set(cl);
   }

   @Override
   protected Class<Function> compile(String source, String sourceType)
throws FunctionException {

       // Extreme hackery:
       // This is using the scala interpreter to compile scala code.
       // The interpreter doesn't expose the class loader it uses so code
is injected into
       // the code to interpret to return the class through a thread local
variable.
       // This also requires trying to figure out which class was meant to
be defined using
       // a regex.

       Matcher matcher = findClasses.matcher(source);
       if (!matcher.find()) {
           throw new FunctionException("Could not find a defined class.");
       }
       String className = matcher.group(1);

       Settings settings = new Settings();
       Interpreter interpreter = new Interpreter(settings);

       lastDefinedClass.set(null);
       interpreter.interpret(
           source + "\n" +
           ScalaFunctionFactory.class.getName() +
".notifyDefinedClass(classOf[" + className + "])\n"
       );

       Class clazz = lastDefinedClass.get();
       if (clazz == null) {
           throw new FunctionException("No class defined in source.");
       }

       return clazz;

   }

}



--
View this message in context: http://old.nabble.com/Compiling-a-Scala-Snippet-at-run-time-tp25397478p27171318.html
Sent from the Scala mailing list archive at Nabble.com.


Loading...