Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: nodejs/node
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 7af8e20ee8e96b3d1ae90e9d5195dcc828a72418
Choose a base ref
...
head repository: nodejs/node
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: b3041a0838d2a329e4239430cadf2160b9fda1a8
Choose a head ref
  • 6 commits
  • 33 files changed
  • 2 contributors

Commits on Dec 20, 2023

  1. src: make process binding data weak

    Avoid the realm being strongly referenced by the process binding data.
    
    PR-URL: #48655
    Backport-PR-URL: #51239
    Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
    Reviewed-By: James M Snell <jasnell@gmail.com>
    Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
    legendecas authored and UlisesGascon committed Dec 20, 2023

    Verified

    This commit was signed with the committer’s verified signature.
    UlisesGascon Ulises GascΓ³n
    Copy the full SHA
    444affd View commit details
  2. src: create worker per isolate properties

    PR-URL: #48655
    Backport-PR-URL: #51239
    Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
    Reviewed-By: James M Snell <jasnell@gmail.com>
    Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
    legendecas authored and UlisesGascon committed Dec 20, 2023

    Verified

    This commit was signed with the committer’s verified signature.
    UlisesGascon Ulises GascΓ³n
    Copy the full SHA
    eb7170c View commit details
  3. src: create fs_dir per isolate properties

    PR-URL: #48655
    Backport-PR-URL: #51239
    Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
    Reviewed-By: James M Snell <jasnell@gmail.com>
    Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
    legendecas authored and UlisesGascon committed Dec 20, 2023

    Verified

    This commit was signed with the committer’s verified signature.
    UlisesGascon Ulises GascΓ³n
    Copy the full SHA
    6b9892b View commit details
  4. src: create per isolate proxy env template

    PR-URL: #48655
    Backport-PR-URL: #51239
    Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
    Reviewed-By: James M Snell <jasnell@gmail.com>
    Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
    legendecas authored and UlisesGascon committed Dec 20, 2023

    Verified

    This commit was signed with the committer’s verified signature.
    UlisesGascon Ulises GascΓ³n
    Copy the full SHA
    23a86e5 View commit details
  5. module: remove useCustomLoadersIfPresent flag

    The flag is always true and can be determined by isLoaderWorker solely.
    
    PR-URL: #48655
    Backport-PR-URL: #51239
    Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
    Reviewed-By: James M Snell <jasnell@gmail.com>
    Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
    legendecas authored and UlisesGascon committed Dec 20, 2023

    Verified

    This commit was signed with the committer’s verified signature.
    UlisesGascon Ulises GascΓ³n
    Copy the full SHA
    ec63a13 View commit details
  6. module: bootstrap module loaders in shadow realm

    This bootstraps ESM loaders in the ShadowRealm with
    `ShadowRealm.prototype.importValue` as its entry point and enables
    loading ESM and CJS modules in the ShadowRealm. The module is imported
    without a parent URL and resolved with the current process's working
    directory.
    
    PR-URL: #48655
    Backport-PR-URL: #51239
    Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
    Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
    Reviewed-By: James M Snell <jasnell@gmail.com>
    Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
    legendecas authored and UlisesGascon committed Dec 20, 2023

    Verified

    This commit was signed with the committer’s verified signature.
    UlisesGascon Ulises GascΓ³n
    Copy the full SHA
    b3041a0 View commit details
Showing with 567 additions and 209 deletions.
  1. +11 βˆ’2 lib/internal/bootstrap/realm.js
  2. +21 βˆ’0 lib/internal/bootstrap/shadow_realm.js
  3. +1 βˆ’0 lib/internal/main/worker_thread.js
  4. +5 βˆ’6 lib/internal/modules/esm/loader.js
  5. +33 βˆ’10 lib/internal/modules/esm/utils.js
  6. +2 βˆ’2 lib/internal/process/esm_loader.js
  7. +2 βˆ’3 lib/internal/process/per_thread.js
  8. +30 βˆ’8 lib/internal/process/pre_execution.js
  9. +13 βˆ’9 src/async_wrap.cc
  10. +8 βˆ’4 src/env.cc
  11. +101 βˆ’81 src/module_wrap.cc
  12. +8 βˆ’5 src/module_wrap.h
  13. +5 βˆ’2 src/node_binding.h
  14. +16 βˆ’12 src/node_dir.cc
  15. +2 βˆ’1 src/node_env_var.cc
  16. +4 βˆ’0 src/node_errors.h
  17. +40 βˆ’30 src/node_messaging.cc
  18. +1 βˆ’1 src/node_messaging.h
  19. +12 βˆ’8 src/node_process.h
  20. +39 βˆ’23 src/node_process_methods.cc
  21. +3 βˆ’2 src/node_realm.cc
  22. +34 βˆ’0 src/node_shadow_realm.cc
  23. +15 βˆ’0 test/fixtures/es-module-shadow-realm/custom-loaders.js
  24. +9 βˆ’0 test/fixtures/es-module-shadow-realm/preload-main.js
  25. +1 βˆ’0 test/fixtures/es-module-shadow-realm/preload.js
  26. +3 βˆ’0 test/fixtures/es-module-shadow-realm/re-export-state-counter.mjs
  27. +4 βˆ’0 test/fixtures/es-module-shadow-realm/state-counter.mjs
  28. +21 βˆ’0 test/parallel/test-shadow-realm-allowed-builtin-modules.js
  29. +26 βˆ’0 test/parallel/test-shadow-realm-custom-loaders.js
  30. +20 βˆ’0 test/parallel/test-shadow-realm-gc-module.js
  31. +28 βˆ’0 test/parallel/test-shadow-realm-import-value-resolve.js
  32. +29 βˆ’0 test/parallel/test-shadow-realm-module.js
  33. +20 βˆ’0 test/parallel/test-shadow-realm-preload-module.js
13 changes: 11 additions & 2 deletions lib/internal/bootstrap/realm.js
Original file line number Diff line number Diff line change
@@ -50,6 +50,8 @@

const {
ArrayFrom,
ArrayPrototypeFilter,
ArrayPrototypeIncludes,
ArrayPrototypeMap,
ArrayPrototypePush,
ArrayPrototypeSlice,
@@ -215,8 +217,8 @@ const internalBuiltinIds = builtinIds
.filter((id) => StringPrototypeStartsWith(id, 'internal/') && id !== selfId);

// When --expose-internals is on we'll add the internal builtin ids to these.
const canBeRequiredByUsersList = new SafeSet(publicBuiltinIds);
const canBeRequiredByUsersWithoutSchemeList =
let canBeRequiredByUsersList = new SafeSet(publicBuiltinIds);
let canBeRequiredByUsersWithoutSchemeList =
new SafeSet(publicBuiltinIds.filter((id) => !schemelessBlockList.has(id)));

/**
@@ -269,6 +271,13 @@ class BuiltinModule {
}
}

static setRealmAllowRequireByUsers(ids) {
canBeRequiredByUsersList =
new SafeSet(ArrayPrototypeFilter(ids, (id) => ArrayPrototypeIncludes(publicBuiltinIds, id)));
canBeRequiredByUsersWithoutSchemeList =
new SafeSet(ArrayPrototypeFilter(ids, (id) => !schemelessBlockList.has(id)));
}

// To be called during pre-execution when --expose-internals is on.
// Enables the user-land module loader to access internal modules.
static exposeInternals() {
21 changes: 21 additions & 0 deletions lib/internal/bootstrap/shadow_realm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
'use strict';

// This script sets up the context for shadow realms.

const {
prepareShadowRealmExecution,
} = require('internal/process/pre_execution');
const {
BuiltinModule,
} = require('internal/bootstrap/realm');

BuiltinModule.setRealmAllowRequireByUsers([
/**
* The built-in modules exposed in the ShadowRealm must each be providing
* platform capabilities with no authority to cause side effects such as
* I/O or mutation of values that are shared across different realms within
* the same Node.js environment.
*/
]);

prepareShadowRealmExecution();
1 change: 1 addition & 0 deletions lib/internal/main/worker_thread.js
Original file line number Diff line number Diff line change
@@ -136,6 +136,7 @@ port.on('message', (message) => {
const isLoaderWorker =
doEval === 'internal' &&
filename === require('internal/modules/esm/utils').loaderWorkerId;
// Disable custom loaders in loader worker.
setupUserModules(isLoaderWorker);

if (!hasStdin)
11 changes: 5 additions & 6 deletions lib/internal/modules/esm/loader.js
Original file line number Diff line number Diff line change
@@ -524,15 +524,14 @@ let emittedLoaderFlagWarning = false;
* A loader instance is used as the main entry point for loading ES modules. Currently, this is a singleton; there is
* only one used for loading the main module and everything in its dependency graph, though separate instances of this
* class might be instantiated as part of bootstrap for other purposes.
* @param {boolean} useCustomLoadersIfPresent If the user has provided loaders via the --loader flag, use them.
* @returns {ModuleLoader}
*/
function createModuleLoader(useCustomLoadersIfPresent = true) {
function createModuleLoader() {
let customizations = null;
if (useCustomLoadersIfPresent &&
// Don't spawn a new worker if we're already in a worker thread created by instantiating CustomizedModuleLoader;
// doing so would cause an infinite loop.
!require('internal/modules/esm/utils').isLoaderWorker()) {
// Don't spawn a new worker if custom loaders are disabled. For instance, if
// we're already in a worker thread created by instantiating
// CustomizedModuleLoader; doing so would cause an infinite loop.
if (!require('internal/modules/esm/utils').forceDefaultLoader()) {
const userLoaderPaths = getOptionValue('--experimental-loader');
if (userLoaderPaths.length > 0) {
if (!emittedLoaderFlagWarning) {
43 changes: 33 additions & 10 deletions lib/internal/modules/esm/utils.js
Original file line number Diff line number Diff line change
@@ -4,6 +4,7 @@ const {
ArrayIsArray,
SafeSet,
SafeWeakMap,
Symbol,
ObjectFreeze,
} = primordials;

@@ -157,6 +158,26 @@ function registerModule(referrer, registry) {
moduleRegistries.set(idSymbol, registry);
}

/**
* Registers the ModuleRegistry for dynamic import() calls with a realm
* as the referrer. Similar to {@link registerModule}, but this function
* generates a new id symbol instead of using the one from the referrer
* object.
* @param {globalThis} globalThis The globalThis object of the realm.
* @param {ModuleRegistry} registry
*/
function registerRealm(globalThis, registry) {
let idSymbol = globalThis[host_defined_option_symbol];
// If the per-realm host-defined options is already registered, do nothing.
if (idSymbol) {
return;
}
// Otherwise, register the per-realm host-defined options.
idSymbol = Symbol('Realm globalThis');
globalThis[host_defined_option_symbol] = idSymbol;
moduleRegistries.set(idSymbol, registry);
}

/**
* Defines the `import.meta` object for a given module.
* @param {symbol} symbol - Reference to the module.
@@ -192,28 +213,29 @@ async function importModuleDynamicallyCallback(referrerSymbol, specifier, attrib
throw new ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING();
}

let _isLoaderWorker = false;
let _forceDefaultLoader = false;
/**
* Initializes handling of ES modules.
* This is configured during pre-execution. Specifically it's set to true for
* the loader worker in internal/main/worker_thread.js.
* @param {boolean} [isLoaderWorker=false] - A boolean indicating whether the loader is a worker or not.
* @param {boolean} [forceDefaultLoader=false] - A boolean indicating disabling custom loaders.
*/
function initializeESM(isLoaderWorker = false) {
_isLoaderWorker = isLoaderWorker;
function initializeESM(forceDefaultLoader = false) {
_forceDefaultLoader = forceDefaultLoader;
initializeDefaultConditions();
// Setup per-isolate callbacks that locate data or callbacks that we keep
// Setup per-realm callbacks that locate data or callbacks that we keep
// track of for different ESM modules.
setInitializeImportMetaObjectCallback(initializeImportMetaObject);
setImportModuleDynamicallyCallback(importModuleDynamicallyCallback);
}

/**
* Determine whether the current process is a loader worker.
* @returns {boolean} Whether the current process is a loader worker.
* Determine whether custom loaders are disabled and it is forced to use the
* default loader.
* @returns {boolean}
*/
function isLoaderWorker() {
return _isLoaderWorker;
function forceDefaultLoader() {
return _forceDefaultLoader;
}

/**
@@ -253,10 +275,11 @@ async function initializeHooks() {

module.exports = {
registerModule,
registerRealm,
initializeESM,
initializeHooks,
getDefaultConditions,
getConditionsSet,
loaderWorkerId: 'internal/modules/esm/worker',
isLoaderWorker,
forceDefaultLoader,
};
4 changes: 2 additions & 2 deletions lib/internal/process/esm_loader.js
Original file line number Diff line number Diff line change
@@ -11,10 +11,10 @@ let esmLoader;

module.exports = {
get esmLoader() {
return esmLoader ??= createModuleLoader(true);
return esmLoader ??= createModuleLoader();
},
async loadESM(callback) {
esmLoader ??= createModuleLoader(true);
esmLoader ??= createModuleLoader();
try {
const userImports = getOptionValue('--import');
if (userImports.length > 0) {
5 changes: 2 additions & 3 deletions lib/internal/process/per_thread.js
Original file line number Diff line number Diff line change
@@ -28,7 +28,6 @@ const {
StringPrototypeStartsWith,
Symbol,
SymbolIterator,
Uint32Array,
} = primordials;

const {
@@ -65,10 +64,10 @@ function refreshHrtimeBuffer() {
// The 3 entries filled in by the original process.hrtime contains
// the upper/lower 32 bits of the second part of the value,
// and the remaining nanoseconds of the value.
hrValues = new Uint32Array(binding.hrtimeBuffer);
hrValues = binding.hrtimeBuffer;
// Use a BigUint64Array in the closure because this is actually a bit
// faster than simply returning a BigInt from C++ in V8 7.1.
hrBigintValues = new BigUint64Array(binding.hrtimeBuffer, 0, 1);
hrBigintValues = new BigUint64Array(binding.hrtimeBuffer.buffer, 0, 1);
}

// Create the buffers.
38 changes: 30 additions & 8 deletions lib/internal/process/pre_execution.js
Original file line number Diff line number Diff line change
@@ -67,6 +67,26 @@ function prepareWorkerThreadExecution() {
});
}

function prepareShadowRealmExecution() {
const { registerRealm } = require('internal/modules/esm/utils');
// Patch the process object with legacy properties and normalizations.
// Do not expand argv1 as it is not available in ShadowRealm.
patchProcessObject(false);
setupDebugEnv();

// Disable custom loaders in ShadowRealm.
setupUserModules(true);
registerRealm(globalThis, {
__proto__: null,
importModuleDynamically: (specifier, _referrer, attributes) => {
// The handler for `ShadowRealm.prototype.importValue`.
const { esmLoader } = require('internal/process/esm_loader');
// `parentURL` is not set in the case of a ShadowRealm top-level import.
return esmLoader.import(specifier, undefined, attributes);
},
});
}

function prepareExecution(options) {
const { expandArgv1, initializeModules, isMainThread } = options;

@@ -160,16 +180,17 @@ function setupSymbolDisposePolyfill() {
}
}

function setupUserModules(isLoaderWorker = false) {
function setupUserModules(forceDefaultLoader = false) {
initializeCJSLoader();
initializeESMLoader(isLoaderWorker);
initializeESMLoader(forceDefaultLoader);
const CJSLoader = require('internal/modules/cjs/loader');
assert(!CJSLoader.hasLoadedAnyUserCJSModule);
// Loader workers are responsible for doing this themselves.
if (isLoaderWorker) {
return;
// Do not enable preload modules if custom loaders are disabled.
// For example, loader workers are responsible for doing this themselves.
// And preload modules are not supported in ShadowRealm as well.
if (!forceDefaultLoader) {
loadPreloadModules();
}
loadPreloadModules();
// Need to be done after --require setup.
initializeFrozenIntrinsics();
}
@@ -687,9 +708,9 @@ function initializeCJSLoader() {
initializeCJS();
}

function initializeESMLoader(isLoaderWorker) {
function initializeESMLoader(forceDefaultLoader) {
const { initializeESM } = require('internal/modules/esm/utils');
initializeESM(isLoaderWorker);
initializeESM(forceDefaultLoader);

// Patch the vm module when --experimental-vm-modules is on.
// Please update the comments in vm.js when this block changes.
@@ -765,6 +786,7 @@ module.exports = {
setupUserModules,
prepareMainThreadExecution,
prepareWorkerThreadExecution,
prepareShadowRealmExecution,
markBootstrapComplete,
loadPreloadModules,
initializeFrozenIntrinsics,
22 changes: 13 additions & 9 deletions src/async_wrap.cc
Original file line number Diff line number Diff line change
@@ -372,8 +372,9 @@ void AsyncWrap::CreatePerContextProperties(Local<Object> target,
Local<Value> unused,
Local<Context> context,
void* priv) {
Environment* env = Environment::GetCurrent(context);
Isolate* isolate = env->isolate();
Realm* realm = Realm::GetCurrent(context);
Environment* env = realm->env();
Isolate* isolate = realm->isolate();
HandleScope scope(isolate);

PropertyAttribute ReadOnlyDontDelete =
@@ -446,13 +447,16 @@ void AsyncWrap::CreatePerContextProperties(Local<Object> target,

#undef FORCE_SET_TARGET_FIELD

env->set_async_hooks_init_function(Local<Function>());
env->set_async_hooks_before_function(Local<Function>());
env->set_async_hooks_after_function(Local<Function>());
env->set_async_hooks_destroy_function(Local<Function>());
env->set_async_hooks_promise_resolve_function(Local<Function>());
env->set_async_hooks_callback_trampoline(Local<Function>());
env->set_async_hooks_binding(target);
// TODO(legendecas): async hook functions are not realm-aware yet.
// This simply avoid overriding principal realm's functions when a
// ShadowRealm initializes the binding.
realm->set_async_hooks_init_function(Local<Function>());
realm->set_async_hooks_before_function(Local<Function>());
realm->set_async_hooks_after_function(Local<Function>());
realm->set_async_hooks_destroy_function(Local<Function>());
realm->set_async_hooks_promise_resolve_function(Local<Function>());
realm->set_async_hooks_callback_trampoline(Local<Function>());
realm->set_async_hooks_binding(target);
}

void AsyncWrap::RegisterExternalReferences(
12 changes: 8 additions & 4 deletions src/env.cc
Original file line number Diff line number Diff line change
@@ -504,6 +504,7 @@ void IsolateData::CreateProperties() {
binding::CreateInternalBindingTemplates(this);

contextify::ContextifyContext::InitializeGlobalTemplates(this);
CreateEnvProxyTemplate(this);
}

constexpr uint16_t kDefaultCppGCEmebdderID = 0x90de;
@@ -1653,10 +1654,13 @@ void AsyncHooks::MemoryInfo(MemoryTracker* tracker) const {
void AsyncHooks::grow_async_ids_stack() {
async_ids_stack_.reserve(async_ids_stack_.Length() * 3);

env()->async_hooks_binding()->Set(
env()->context(),
env()->async_ids_stack_string(),
async_ids_stack_.GetJSArray()).Check();
env()
->principal_realm()
->async_hooks_binding()
->Set(env()->context(),
env()->async_ids_stack_string(),
async_ids_stack_.GetJSArray())
.Check();
}

void AsyncHooks::FailWithCorruptedAsyncStack(double expected_async_id) {
Loading