Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build] Can not build 1.17.1 for NodeJS #20082

Closed
pbk20191 opened this issue Mar 26, 2024 · 14 comments
Closed

[Build] Can not build 1.17.1 for NodeJS #20082

pbk20191 opened this issue Mar 26, 2024 · 14 comments
Labels
build build issues; typically submitted using template ep:CUDA issues related to the CUDA execution provider ep:DML issues related to the DirectML execution provider ep:TensorRT issues related to TensorRT execution provider platform:web issues related to ONNX Runtime web; typically submitted using template platform:windows issues related to the Windows platform

Comments

@pbk20191
Copy link

Describe the issue

I can build for cpu only but I can not complete the build for other EPs. I can not find what is going wrong, cause the log gives me too few infomations.

build command just fails with non zero exit code. After the build I can install npm package from source with "onnxruntime_binding.node", "DirectML.dll", "onnxruntime.dll", but "onnxruntime_providers_shared.dll" is missing.

I guess there is something goes wrong right after making node binding?

Urgency

No response

Target platform

Window 11(x64), cuda 12.4, cudnn 9.0 (12.3)

Build script

.\build.bat --config Release --parallel --build_nodejs --use_dml --use_cuda --use_tensorrt --tensorrt_home "C:\SystemSource\NVIDIA GPU Computing Toolkit\TensorRT\8.6.1.6" --build_shared_lib

Error / output

Using cmake-js to build OnnxRuntime Node.js binding
  
  > onnxruntime-node@1.17.1 preprepare
  > node -e "require('node:fs').copyFileSync('./node_modules/long/index.d.ts', './node_modules/long/umd/index.d
  .ts')"

  
  > onnxruntime-node@1.17.1 prepare
  > tsc --build script test .

  
  added 95 packages, and audited 97 packages in 4s

  6 packages are looking for funding
    run `npm fund` for details

  1 moderate severity vulnerability

  To address all issues, run:
    npm audit fix

  Run `npm audit` for details.
  
  > onnxruntime-node@1.17.1 build
  > tsc && node ./script/build --onnxruntime-build-dir=C:/Users/pbk/Documents/VSCode/onnxruntime/build/Windows/
  Release --config=Release --arch=x64 --use_cuda --use_dml --use_tensorrt

  info find VS using VS2022 (17.9.34714.143) found at:
  info find VS "C:\Program Files\Microsoft Visual Studio\2022\Community"
  info find VS run with --verbose for detailed information
  info TOOL Using Visual Studio 17 2022 generator.
  info CMD CONFIGURE
  info find VS using VS2022 (17.9.34714.143) found at:
  info find VS "C:\Program Files\Microsoft Visual Studio\2022\Community"
  info find VS run with --verbose for detailed information
  info RUN [
  info RUN   'cmake',
  info RUN   'C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node',
  info RUN   '--no-warn-unused-cli',
  info RUN   '-G',
  info RUN   'Visual Studio 17 2022',
  info RUN   '-A',
  info RUN   'x64',
  info RUN   '-DCMAKE_JS_VERSION=7.2.1',
  info RUN   '-DCMAKE_BUILD_TYPE=Release',
  info RUN   '-DCMAKE_RUNTIME_OUTPUT_DIRECTORY=C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node\\build'
  ,
  info RUN   '-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded$<$<CONFIG:Debug>:Debug>',
  info RUN   '-DCMAKE_JS_INC=C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node\\node_modules\\node-api-h
  eaders\\include;C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node\\node_modules\\node-addon-api',
  info RUN   '-DCMAKE_JS_SRC=C:/Users/pbk/Documents/VSCode/onnxruntime/js/node/node_modules/cmake-js/lib/cpp/wi
  n_delay_load_hook.cc',
  info RUN   '-DNODE_RUNTIME=node',
  info RUN   '-DNODE_RUNTIMEVERSION=21.7.1',
  info RUN   '-DNODE_ARCH=x64',
  info RUN   '-Dnapi_build_version=6',
  info RUN   '-DCMAKE_BUILD_TYPE=Release',
  info RUN   '-DONNXRUNTIME_BUILD_DIR=C:/Users/pbk/Documents/VSCode/onnxruntime/build/Windows/Release',
  info RUN   '-DUSE_DML=ON',
  info RUN   '-DUSE_CUDA=ON',
  info RUN   '-DUSE_TENSORRT=ON',
  info RUN   '-DCMAKE_JS_LIB=C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node\\build\\node.lib',
  info RUN   '-DCMAKE_JS_NODELIB_DEF=C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node\\build\\node-lib.
  def',
  info RUN   '-DCMAKE_JS_NODELIB_TARGET=C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\js\\node\\build\\node.l
  ib',
  info RUN   '-DCMAKE_SHARED_LINKER_FLAGS=/DELAYLOAD:NODE.EXE'
  info RUN ]
  Not searching for unused variables given on the command line.
  -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22631.
  Microsoft (R) Library Manager Version 14.39.33523.0
  Copyright (C) Microsoft Corporation.  All rights reserved.

     C:\Users\pbk\Documents\VSCode\onnxruntime\js\node\build\node.lib ?쇱씠釉뚮윭由?諛?C:\Users\pbk\Documents\VSCode\o
  nnxruntime\js\node\build\node.exp 媛쒖껜瑜??앹꽦?섍퀬 ?덉뒿?덈떎.
  -- Configuring done (0.2s)
  -- Generating done (0.1s)
  -- Build files have been written to: C:/Users/pbk/Documents/VSCode/onnxruntime/js/node/build
  msbuild 버전 17.9.8+b34f75857(.NET Framework용)
  
    1>Checking Build System
    Building Custom Rule C:/Users/pbk/Documents/VSCode/onnxruntime/js/node/CMakeLists.txt
    directml_load_helper.cc
    inference_session_wrap.cc
    main.cc
    run_options_helper.cc
    session_options_helper.cc
    tensor_helper.cc
    win_delay_load_hook.cc
    코드를 생성하고 있습니다...
       C:/Users/pbk/Documents/VSCode/onnxruntime/js/node/build/Release/onnxruntime_binding.lib 라이브러리 및 C:/Users
  /pbk/Documents/VSCode/onnxruntime/js/node/build/Release/onnxruntime_binding.exp 개체를 생성하고 있습니다.
    onnxruntime_binding.vcxproj -> C:\Users\pbk\Documents\VSCode\onnxruntime\js\node\build\Release\onnxruntime_
  binding.node
    Building Custom Rule C:/Users/pbk/Documents/VSCode/onnxruntime/js/node/CMakeLists.txt
  all good ---------------------------------------- 
  Building Custom Rule C:/Users/pbk/Documents/VSCode/onnxruntime/cmake/CMakeLists.txt
Traceback (most recent call last):
  File "C:\Users\pbk\Documents\VSCode\onnxruntime\tools\ci_build\build.py", line 2890, in <module>
    sys.exit(main())
  File "C:\Users\pbk\Documents\VSCode\onnxruntime\tools\ci_build\build.py", line 2782, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "C:\Users\pbk\Documents\VSCode\onnxruntime\tools\ci_build\build.py", line 1662, in build_targets
    run_subprocess(cmd_args, env=env)
  File "C:\Users\pbk\Documents\VSCode\onnxruntime\tools\ci_build\build.py", line 839, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "C:\Users\pbk\Documents\VSCode\onnxruntime\tools\python\util\run.py", line 49, in run
    completed_process = subprocess.run(
  File "C:\Users\pbk\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\.venv\\Scripts\\cmake.EXE', '--build', 'C:\\Users\\pbk\\Documents\\VSCode\\onnxruntime\\\\build\\Windows\\Release', '--config', 'Release', '--', '/maxcpucount:24', '/nodeReuse:False', '/p:CL_MPCount=24']' returned non-zero exit status 1.

Visual Studio Version

2022 17 community

GCC / Compiler Version

No response

@pbk20191 pbk20191 added the build build issues; typically submitted using template label Mar 26, 2024
@github-actions github-actions bot added ep:CUDA issues related to the CUDA execution provider ep:DML issues related to the DirectML execution provider ep:TensorRT issues related to TensorRT execution provider platform:web issues related to ONNX Runtime web; typically submitted using template platform:windows issues related to the Windows platform labels Mar 26, 2024
@fs-eire
Copy link
Contributor

fs-eire commented Mar 28, 2024

There are some code changes and we are including them in 1.17.3. this patch release will be out soon

@nemphys
Copy link

nemphys commented Apr 5, 2024

Any news on this? I have managed to build the current master from source (onnxruntime-node), but using npm we are still at 1.17.0.

@nemphys
Copy link

nemphys commented Apr 18, 2024

Now that 1.17.3 has been finally released, onnxrutime-web is on 1.17.3 but onnxruntime-node is still on 1.17.0...
Is there something wrong with the npm publishing pipeline?

@fs-eire
Copy link
Contributor

fs-eire commented Apr 18, 2024

Now that 1.17.3 has been finally released, onnxrutime-web is on 1.17.3 but onnxruntime-node is still on 1.17.0... Is there something wrong with the npm publishing pipeline?

The package tarball is generated successfully, however its size is too large and is rejected by NPM registery.

I did this change to main branch to fix this issue.

I also created a PR for the same change on branch rel-1.17.3 but a test failure is blocking the merge. I am trying to find a way to workaround this.

@nemphys
Copy link

nemphys commented Apr 18, 2024

This sounds great! But does it mean that we will have to wait for the next release for it to be actually usable?

@fs-eire
Copy link
Contributor

fs-eire commented Apr 18, 2024

This sounds great! But does it mean that we will have to wait for the next release for it to be actually usable?

I am still trying to get the change into 1.17.3....

UPDATE: onnxruntime-node@1.17.3 is published. @nemphys

@nemphys
Copy link

nemphys commented Apr 20, 2024

Just tried it and I unfortunately get this error:

npm install onnxruntime-node --onnxruntime-node-install-cuda=v12

npm ERR! code 1
npm ERR! path /Users/user/git/node/onnx/node_modules/onnxruntime-node
npm ERR! command failed
npm ERR! command sh -c node ./script/install
npm ERR! Downloading "https://github.com/microsoft/onnxruntime/releases/download/v1.17.3/onnxruntime-linux-x64-cuda12-1.17.3.tgz"...
npm ERR! /Users/user/git/node/onnx/node_modules/onnxruntime-node/script/install.js:67
npm ERR!     throw new Error(`Failed to download the binaries: ${res.status} ${res.statusText}.
npm ERR!           ^
npm ERR!
npm ERR! Error: Failed to download the binaries: 404 Not Found.
npm ERR!
npm ERR! Use "--onnxruntime-node-install-cuda=skip" to skip the installation. You will still be able to use ONNX Runtime, but the CUDA EP will not be available.
npm ERR!     at /Users/user/git/node/onnx/node_modules/onnxruntime-node/script/install.js:67:11
npm ERR!     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
npm ERR!
npm ERR! Node.js v20.12.1

Apparently the CUDA v12 archive URL is not valid.

@nemphys
Copy link

nemphys commented Apr 20, 2024

It seems that the correct filename is onnxruntime-linux-x64-gpu-cuda12-1.17.3.tgz (the "-gpu" part is missing)

@pbk20191
Copy link
Author

Can not create InferenceSession with onnxruntime 1.17.3

const session = onnxruntime.InferenceSession.create(
    "mnist-12.onnx", 
    {
        "executionProviders": [
            {
                name:"tensorrt"
            },
            {
                name: "cuda"
            },
            {
                name: "cpu"
            }
        ]
    }
)
/home/pbk/git-projects/nodefoo/node_modules/onnxruntime-node/lib/backend.ts:14
      this.#inferenceSession.loadModel(pathOrBuffer, options);
                             ^
Error: /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1158 void onnxruntime::ProviderSharedLibrary::Ensure() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_shared.so with error: /home/pbk/git-projects/nodefoo/node_modules/onnxruntime-node/bin/napi-v3/linux/x64/libonnxruntime_providers_shared.so: file too short

    at new OnnxruntimeSessionHandler (/home/pbk/git-projects/nodefoo/node_modules/onnxruntime-node/lib/backend.ts:14:30)
    at Immediate.<anonymous> (/home/pbk/git-projects/nodefoo/node_modules/onnxruntime-node/lib/backend.ts:61:19)
    at processImmediate (node:internal/timers:478:21)

@nemphys
Copy link

nemphys commented Apr 20, 2024

I just tried the new version on a linux x64 machine and (without any cuda-related parameter) the installation succeeds, probably downloading the default cuda11 binaries (the specific server has both cuda 11 and 12 installed, no idea why 11 is selected).

The problem is that the file libonnxruntime_providers_shared.so has a size of 0 bytes (empty) and therefore the error @pbk20191 mentions above arises during runtime.

Apart from that, I think that having to specify a runtime parameter during npm install in order to set the desired preference regarding cuda version/skip is suboptimal. It would be much better to be able to somehow define this preference inside the parent application's package.json, so that a simple npm install would perform the desired operation without any extra/custom parameters. Our app is automatically deployed to various servers (of different architectures) using ansible followed by a call to npm install, so defining runtime parameters is not something we would like to include in out deployment flow.

@fs-eire
Copy link
Contributor

fs-eire commented Apr 20, 2024

Thank you for the feedbacks. A few fixes are on the way at #20397 and a reversion release is being worked on.

UPDATE: Please try onnxruntime-node@1.17.3-rev.1 and let me know if it fixes the issues. @nemphys @pbk20191

@nemphys
Copy link

nemphys commented Apr 21, 2024

@fs-eire I confirm that the new rev.1 works fine without any issues!

@pbk20191
Copy link
Author

onnxruntime-node@1.17.3-rev.1 seems okay to me. It works fine.

@pbk20191
Copy link
Author

Closing the issue since, it is resolved from new release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build issues; typically submitted using template ep:CUDA issues related to the CUDA execution provider ep:DML issues related to the DirectML execution provider ep:TensorRT issues related to TensorRT execution provider platform:web issues related to ONNX Runtime web; typically submitted using template platform:windows issues related to the Windows platform
Projects
None yet
Development

No branches or pull requests

3 participants