https://nebula-graph.io logo
Join Slack
Powered by
Hi everyone. I ran into a problem starting nebula ...
# nebula
d
Hi everyone. I ran into a problem starting nebula graph. metad and storaged always exit immediately due to some errors.
Copy code
F20221124 17:04:31.736765 14793 DiskManager.cpp:39] DataPath invalid: boost::filesystem::status: Function not implemented: "/usr/local/nebula/data/meta"
*** Check failure stack trace: ***
*** Aborted at 1669305871 (Unix time, try 'date -d @1669305871') ***
*** Signal 6 (SIGABRT) (0x39c9) received by PID 14793 (pthread TID 0x7ff1eb7bc080) (linux TID 14793) (maybe from PID 14793, UID 0) (code: -6), stack trace: ***
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x253ad41]
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x26)[0x2532246]
/usr/local/nebula/bin/nebula-metad[0x25301d7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x1297f)[0x7ff1eabdb97f]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7ff1ea816e87]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x140)[0x7ff1ea8187f0]
/usr/local/nebula/bin/nebula-metad[0x2563a49]
/usr/local/nebula/bin/nebula-metad[0x2566b43]
/usr/local/nebula/bin/nebula-metad[0x2563518]
/usr/local/nebula/bin/nebula-metad[0x25671f8]
/usr/local/nebula/bin/nebula-metad[0x1056fa4]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula7kvstore11NebulaStore4initEv+0x2d9)[0x1b46319]
/usr/local/nebula/bin/nebula-metad(_Z6initKVSt6vectorIN6nebula8HostAddrESaIS1_EES1_+0x682)[0x1154562]
/usr/local/nebula/bin/nebula-metad(main+0x8e2)[0x1105532]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe6)[0x7ff1ea7f9c86]
/usr/local/nebula/bin/nebula-metad(_start+0x29)[0x1142cd9]
I checked and there's actually no nebula/data folder at all. am I missing some steps here?
and for storaged
Copy code
terminate called after throwing an instance of 'apache::thrift::concurrency::SystemResourceException'
  what():  pthread_create failed
*** Aborted at 1669303920 (Unix time, try 'date -d @1669303920') ***
*** Signal 6 (SIGABRT) (0x2fd1) received by PID 12241 (pthread TID 0x7f07a8f5c080) (linux TID 12241) (maybe from PID 12241, UID 0) (code: -6), stack trace: ***
/usr/local/nebula/bin/nebula-storaged(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x26443e1]
/usr/local/nebula/bin/nebula-storaged(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x26)[0x263b8e6]
/usr/local/nebula/bin/nebula-storaged[0x2639877]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x1297f)[0x7f07a837b97f]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7f07a7fb6e87]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x140)[0x7f07a7fb87f0]
/usr/local/nebula/bin/nebula-storaged[0x1150f82]
/usr/local/nebula/bin/nebula-storaged(_ZN10__cxxabiv111__terminateEPFvvE+0x5)[0x2b763d5]
/usr/local/nebula/bin/nebula-storaged(_ZSt9terminatev+0x10)[0x2b76440]
/usr/local/nebula/bin/nebula-storaged(__cxa_rethrow+0x45)[0x2b765e5]
/usr/local/nebula/bin/nebula-storaged[0x1108668]
/usr/local/nebula/bin/nebula-storaged(_ZN6apache6thrift11concurrency21PriorityThreadManager12PriorityImpl5startEv+0x5d)[0x230d10d]
/usr/local/nebula/bin/nebula-storaged(_ZN6nebula7storage13StorageServer5startEv+0x19b)[0x11aa73b]
/usr/local/nebula/bin/nebula-storaged(main+0xe64)[0x1154f24]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe6)[0x7f07a7f99c86]
/usr/local/nebula/bin/nebula-storaged(_start+0x29)[0x1191569]
🙂 1
also no idea where to ask any questions, the forum looks abandoned 😕
w
Dear Dominik, welcome to the NeublaGraph community, we archived the forum and switched to Github Discussions(https://github.com/vesoft-inc/nebula/discussions) , sorry for the confusion. This is a known issue(https://github.com/vesoft-inc/nebula/issues/4205) for old Linux kernel versions
Copy code
DataPath invalid: boost::filesystem::status: Function not implemented:
Could you please help share your information here or in the issue? • what is the Linux version, is it ARM64 or? (
uname -a
) Thanks! if possible the mitigations are using modern versions of OS.
d
oh I see, thanks for your help. and I'm on 
Linux / 4.15.0 #1 SMP Thu Apr 22 16:18:59 MSK 2021 x86_64 x86_64 x86_64 GNU/Linux
🙏 1
okay so I'm now on 
Linux / 5.4.0 #1 SMP Thu Apr 22 16:18:59 MSK 2021 x86_64 x86_64 x86_64 GNU/Linux
but I'm constantly running into this in metad:
Copy code
terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable
*** Aborted at 1669568023 (Unix time, try 'date -d @1669568023') ***
*** Signal 6 (SIGABRT) (0x6660) received by PID 26208 (pthread TID 0x7fd4618e6700) (linux TID 26396) (maybe from PID 26208, UID 0) (code: -6), stack trace: ***
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x253ace1]
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x26)[0x25321e6]
/usr/local/nebula/bin/nebula-metad[0x2530177]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x1441f)[0x7fd47e3ab41f]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7fd47e1e800b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12a)[0x7fd47e1c7858]
/usr/local/nebula/bin/nebula-metad[0x1101b02]
/usr/local/nebula/bin/nebula-metad(_ZN10__cxxabiv111__terminateEPFvvE+0x5)[0x2a6c3c5]
/usr/local/nebula/bin/nebula-metad(_ZSt9terminatev+0x10)[0x2a6c430]
/usr/local/nebula/bin/nebula-metad(__cxa_throw+0x43)[0x2a6c583]
/usr/local/nebula/bin/nebula-metad(_ZSt20__throw_system_errori+0x80)[0x1103ae3]
/usr/local/nebula/bin/nebula-metad(_ZNSt6thread15_M_start_threadESt10unique_ptrINS_6_StateESt14default_deleteIS1_EEPFvvE+0x28)[0x2aea238]
/usr/local/nebula/bin/nebula-metad(_ZN5folly18ThreadPoolExecutor10addThreadsEm+0x5a3)[0x2478593]
/usr/local/nebula/bin/nebula-metad(_ZN5folly18ThreadPoolExecutor19ensureActiveThreadsEv+0x84)[0x247d3a4]
/usr/local/nebula/bin/nebula-metad(_ZN5folly18ThreadPoolExecutor11addObserverESt10shared_ptrINS0_8ObserverEE+0x10f)[0x247d8bf]
/usr/local/nebula/bin/nebula-metad(_ZN8proxygen10HTTPServer14startTcpServerESt10shared_ptrIN6wangle15AcceptorFactoryEES1_IN5folly20IOThreadPoolExecutorEE+0x1f7)[0x2357927]
/usr/local/nebula/bin/nebula-metad(_ZN8proxygen10HTTPServer5startESt8functionIFvvEES1_IFvNSt15__exception_ptr13exception_ptrEEESt10shared_ptrIN6wangle15AcceptorFactoryEES8_IN5folly20IOThreadPoolExecutorEE+0xa8)[0x2358988]
/usr/local/nebula/bin/nebula-metad[0x1a462a5]
/usr/local/nebula/bin/nebula-metad[0x1a46017]
/usr/local/nebula/bin/nebula-metad[0x2aea15f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8608)[0x7fd47e39f608]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x42)[0x7fd47e2c4132]
(safe mode, symbolizer not available)
I somehow managed to start everything and connect once, tried to ADD HOSTS which was then still offline, tried to restart and since then I can't do anything really
❤️ 1
w
Dear @Dominik are you still encountering this crash or? The storage hosts to be activated with 
ADD HOSTS
should be exactly the same as it was configured in storaged’s conf for its IP(or host) and Port. The traffic between metad and storaged should be done properly to enable its ONLINE. Could you share the conf and logs of storaged and metad if it’s still offline?
d
127.0.0.1:9779 was what I tried to add as a host I believe. which should be the default I think. I'll add the conf files but I didn't change anything here. storaged error logs are actually empty, metad is always crashing with the above error so nothing else there either. But I noticed that storaged is also very stubborn and stopping it often takes a few 'kill storaged' attempts. I think I will try to clean everything up and redo the entire setup from step 1 tomorrow.
nebula-metad.confnebula-storaged.conf
w
ok, then metad crash is the only issue now. Could you please kindly help report this at https://github.com/vesoft-inc/nebula/issues ? With reproducing conditions/procedures? Thanks! This is quite strange/new to us, I am really curious about its trigger conditions.
d
well, I raised an issue https://github.com/vesoft-inc/nebula/issues/4957 but I don't really have any reproducing steps other than install and run as that's all I'm doing, or I don't know what to add.
❤️ 2
m
Do you have enough disk space?
d
1TB SSD/ 32GB RAM from a specification point of view. unless I have to manually grant it to the services?
123 Views
Open in Slack
PreviousNext
Powered by