define easy, cross-platform way of building addons

[ry]

No description provided.

[paddybyers]

I started work on a .gyp file for building an addon.

One issue is the convention for the name of the shared library. With various toolchains it's not possible to conform to the ".node" extension; and even if it's possible in the platform toolchain, it's not possible to get gyp to emit the necessary configuration in the build file.

Obviously we could get the project file to emit a library whose name satisfies the usual platform convention - eg libhello.dylib instead of hello.node - and then symlink or copy the file as part of an install phase. But it would be nicer if we didn't have to do that, and could instead get node to recognise a library whose name aligns with the platform default.

I know that simply adding a new extension to module.js will increase the size of the search space for modules - which is already excessively big based on the number of paths and extensions to be searched - so we would have to selectively add just the one extension appropriate to the platform (and perhaps not look for .node at all).

Any thoughts from the core team?

[kkaefer]

https://github.com/mapnik/node-mapnik/blob/gyp/build.gyp has a sample gyp file created by @springmeyer

[arturadib]

Could #2337 offer the desired solution?

[dobesv]

I wonder if you could output a .json or even a .js file which specifies the specific path of the module relative to itself? This would eliminate the need to check for additional file extensions (the node docs say it already is looking for a .json file extension) but would add a bit of process to the .json file handling (to specify the path to the native module).

For json it might have a key like "native_modules" with a key for each platform that was compiled tell node where the final binary to load is.

If a .js file were output, that file could use require('') or some other method to specify the full path to the correct binary, and it may also use the process.arch and process.platform to figure out the best file to load. The just re-export everything that was export by the binary module as its own. Or possibly say "exports = require('./modulename.dylib') ?

[dobesv]

Also it would be good if you can put pre-build binaries for multiple architectures into the same folder / search path. See #2414

My suggest solution in the previous comment might help with that at the same time.

[arturadib]

Curious to hear about the roadmap for addon building - I've tried to compile the helloworld addon example using gyp_addon. On my mac, an Xcode project is created, but it fails to build.

Any insights?

[arturadib]

Also, I've noticed that make install installs node-waf and include/ headers, but not gyp_addon. Will it eventually be part of the make install?

[paddybyers]

@arturadib : can you try this patch: paddybyers@e27fd0f

?

The addon.gypi on the mac should work with the make generator, but the above patch should also make it work with the xcode generator.

[isaacs]

gyp_addon will never be a part of the default make install.

Please go follow https://github.com/TooTallNate/node-gyp for more exciting updates on this.

@TooTallNate Can you please post some relevant details on this issue, and the others linking to it? Thanks.

[TooTallNate]

@isaacs Thanks for the introduction!

So node-gyp is the project spawned from the recent thread on the mailing list calling to address this native modules using gyp situation. Having been messing around with native modules on Windows for a little while, I was very excited to have a chance to write this and jumped at the opportunity. I am currently very pleased with the initial result.

node-gyp is designed as an almost drop-in replacement to node-waf so the command-line API will be very familiar. The major difference is that you now need a bindings.gyp file in your module root instead of a wscript file. See the node-gyp README for an example gyp file and some additional resources to get you going there.

As @isaacs already said, this module won't be included with node distributions, and instead the module devs and more advanced users are the ones expected to compile the modules, ideally offering precompiled modules for the new users to benefit from. The goal there is lowering the bar needed to start getting into node.js, while ideally not affecting current/advanced users too much (I mean, we were already compiling modules ourselves, so nothing changes there...).

But overall, let's get your modules converted over! I have started the process of "driving adoption" by sending pull requests to some of the common native modules, so let's continue the trend!

[TooTallNate]

P.S. if anybody would like help converting their native module to gyp just give a shout out and I'll give you priority! I'm still trying to prioritize which modules I am attempting to convert; @isaacs a list of the most depended on native node modules would be nice :)

[gero3]

I am actually trying to convert one. ajaxorg/o3 to be exact

[TooTallNate]

@gero3 Sure thing. Which one?

[gero3]

https://github.com/gero3/o3 is what I have for now. latest commit

[TooTallNate]

@gero3 What you have seems alright to me (not sure about the final LINKFLAGS part, that might happen automatically). But when I try to compile I get a bunch of header files that do not exist, like:

include/cStream.h:30:32: error: cStream_posix_glue.h: No such file or directory

So it seems to me like the module is missing some stuff as-is. Also >40mb download for the repo! Holy moly!

[gero3]

Ah first, you need to run node tools/node_modules_build which builds the glue files.(even it if fails on node-waf). Sorry about that.

PS: I found the verbose option so I'look where the differences in node-waf and node-gyp

PS2: https://gist.github.com/1764377 is the output generated from the build

[isaacs]

@TooTallNate Can we close this issue? It seems like the node-gyp stuff is moving along on its own now.

[TooTallNate]

@isaacs I think that would be appropriate :)

[arturadib]

@TooTallNate ooh look at you all powerful and stuff :) (needless to say, superb work with node-gyp -- it's a keeper)