Is open source nebula-explorer deployment supported?

Is there currently a timeline for releasing 3.2 for nebula-operator?

Hi folks, I am trying to install nebula on ec2 server from this doc. I am facing error

0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling

. after the installation, can someone please help me.

@Luc Juggery Welcome to the NebulaGraph community!😉

Hi Folks, I am trying install nebula-graph3.2.0 on ubuntu server using source code compilation method and seeing dependencies error related to rocksdb. However, I was able to install the same nebula-graph version 3.2.0 successfully few days ago, not sure what changed now. Can someone please take a look and help me with this, Thanks. Error :

/home/ubuntu/nebula/src/kvstore/RocksEngine.h:11:10: fatal error: rocksdb/utilities/backupable_db.h: No such file or directory

Hi Folks, We have setup a nebula cluster on the AWS EKS using kubectl (Ref document : Link) and all the meta,graph and storage services are started successfully. Now, I am trying to setup the

nebula backup & restore (nebula-br)

for this cluster and as part of it, I have started the

nebula-agent

daemon on the EC2 nodes associated with the EKS cluster, but when I am executing the

nebula-br backup

command , it's giving an error

Error: parse cluster response failed: response is not successful, code is E_LIST_CLUSTER_NO_AGENT_FAILURE

. It seems nebula-br is unable to access the nebula-agent process ? Can someone please let us know if this is the correct approach to setup the nebula-agent on kubernates environment ?

Hi, <!channel>, There is a community meeting at 13:30 UTC today (2022/9/7, Please come and join us https://us02web.zoom.us/meeting/register/tZ0rcuypqDMvGdLuIm4VprTlx96wrEf062SH

My v3.2.0 cluster went into a weird state, It's not working. Snapshot send failed, the leader changed? Does anyone know how to fix this? I20220915 162809.498340 102 NebulaSnapshotManager.cpp:67] Space 61 Part 34 start send snapshot of commitLogId 89598562 commitLogTerm 2, rate limited to 10485760, batch size is 1048576 I20220915 162809.498370 102 SnapshotManager.cpp:51] [Port: 9780, Space: 61, Part: 34] Snapshot send failed, the leader changed? I20220915 162809.498400 102 Host.cpp:355] [Port: 9780, Space: 61, Part: 34] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Send snapshot failed! I20220915 162809.498530 53 Host.cpp:337] [Port: 9780, Space: 61, Part: 115] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Can't find log 1 in wal, send the snapshot, logIdToSend = 32549073, firstLogId in wal = 32538198, lastLogId in wal = 32549073 I20220915 162809.498550 66 Host.cpp:337] [Port: 9780, Space: 61, Part: 124] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Can't find log 1 in wal, send the snapshot, logIdToSend = 97588614, firstLogId in wal = 97582907, lastLogId in wal = 97588614 I20220915 162809.498647 103 NebulaSnapshotManager.cpp:67] Space 61 Part 115 start send snapshot of commitLogId 32549062 commitLogTerm 2, rate limited to 10485760, batch size is 1048576 I20220915 162809.498672 103 SnapshotManager.cpp:51] [Port: 9780, Space: 61, Part: 115] Snapshot send failed, the leader changed? I20220915 162809.498690 103 Host.cpp:355] [Port: 9780, Space: 61, Part: 115] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Send snapshot failed! I20220915 162809.498718 104 NebulaSnapshotManager.cpp:67] Space 61 Part 124 start send snapshot of commitLogId 97588603 commitLogTerm 2, rate limited to 10485760, batch size is 1048576 I20220915 162809.498766 104 SnapshotManager.cpp:51] [Port: 9780, Space: 61, Part: 124] Snapshot send failed, the leader changed? I20220915 162809.498817 104 Host.cpp:355] [Port: 9780, Space: 61, Part: 124] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Send snapshot failed! I20220915 162809.499837 72 Host.cpp:337] [Port: 9780, Space: 61, Part: 34] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Can't find log 1 in wal, send the snapshot, logIdToSend = 89598573, firstLogId in wal = 89579895, lastLogId in wal = 89598573 I20220915 162809.499940 101 NebulaSnapshotManager.cpp:67] Space 61 Part 34 start send snapshot of commitLogId 89598562 commitLogTerm 2, rate limited to 10485760, batch size is 1048576 I20220915 162809.499976 101 SnapshotManager.cpp:51] [Port: 9780, Space: 61, Part: 34] Snapshot send failed, the leader changed? I20220915 162809.500000 101 Host.cpp:355] [Port: 9780, Space: 61, Part: 34] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Send snapshot failed! I20220915 162809.500094 53 Host.cpp:337] [Port: 9780, Space: 61, Part: 115] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Can't find log 1 in wal, send the snapshot, logIdToSend = 32549073, firstLogId in wal = 32538198, lastLogId in wal = 32549073 I20220915 162809.500183 102 NebulaSnapshotManager.cpp:67] Space 61 Part 115 start send snapshot of commitLogId 32549062 commitLogTerm 2, rate limited to 10485760, batch size is 1048576 I20220915 162809.500219 102 SnapshotManager.cpp:51] [Port: 9780, Space: 61, Part: 115] Snapshot send failed, the leader changed? I20220915 162809.500241 102 Host.cpp:355] [Port: 9780, Space: 61, Part: 115] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Send snapshot failed! I20220915 162809.500391 66 Host.cpp:337] [Port: 9780, Space: 61, Part: 124] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Can't find log 1 in wal, send the snapshot, logIdToSend = 97588614, firstLogId in wal = 97582907, lastLogId in wal = 97588614 I20220915 162809.500461 103 NebulaSnapshotManager.cpp:67] Space 61 Part 124 start send snapshot of commitLogId 97588603 commitLogTerm 2, rate limited to 10485760, batch size is 1048576 I20220915 162809.500497 103 SnapshotManager.cpp:51] [Port: 9780, Space: 61, Part: 124] Snapshot send failed, the leader changed? I20220915 162809.500515 103 Host.cpp:355] [Port: 9780, Space: 61, Part: 124] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Send snapshot failed! I20220915 162809.501394 72 Host.cpp:337] [Port: 9780, Space: 61, Part: 34] [Host: nebula-cluster-storaged-1.nebula-cluster-storaged-headless.cohort-search.svc.cluster.local:9780] Can't find log 1 in wal, send the snapshot, logIdToSend = 89598573, firstLogId in wal = 89579895, lastLogId in wal = 89598573 I

HI Folks, I am using the nebula-br to backup the data from my nebula-cluster to S3 storage but seeing the below errors. Could you please take a look and help us in resolving this issue, Thanks.

Cleanup backup BACKUP_2022_09_21_18_09_25 successfully after backup failed.Error: upload file by agent failed: rpc error: code = Unknown desc = upload from /usr/local/nebula/data/meta/nebula/0/checkpoints/BACKUP_2022_09_21_18_09_25/__disk_parts__.sst to nebula-backup-restore/BACKUP_2022_09_21_18_09_25/meta/__disk_parts__.sst failed: upload from /usr/local/nebula/data/meta/nebula/0/checkpoints/BACKUP_2022_09_21_18_09_25/__disk_parts__.sst to nebula-backup-restore/BACKUP_2022_09_21_18_09_25/meta/__disk_parts__.sst failed: SerializationError: failed to unmarshal error message
	status code: 405, request id: KMW29S7EGFNE1Y8X, host id: +bWJDdJCflu7yR8AgY7snKHajR4YZyvbYOD2qjuMDG82u+j9GNe9eAd7GVLmlrfrMXMrZPasqQE=
caused by: UnmarshalError: failed to unmarshal error message
	00000000 3c 68 74 6d 6c 3e 0a 3c 68 65 61 64 3e 3c 74 69 |<html>.<head><ti|
00000010 74 6c 65 3e 34 30 35 20 4d 65 74 68 6f 64 20 4e |tle>405 Method N|
00000020 6f 74 20 41 6c 6c 6f 77 65 64 3c 2f 74 69 74 6c |ot Allowed</titl|
00000030 65 3e 3c 2f 68 65 61 64 3e 0a 3c 62 6f 64 79 3e |e></head>.<body>|
00000040 0a 3c 68 31 3e 34 30 35 20 4d 65 74 68 6f 64 20 |.<h1>405 Method |
00000050 4e 6f 74 20 41 6c 6c 6f 77 65 64 3c 2f 68 31 3e |Not Allowed</h1>|
00000060 0a 3c 75 6c 3e 0a 3c 6c 69 3e 43 6f 64 65 3a 20 |.<ul>.<li>Code: |
00000070 4d 65 74 68 6f 64 4e 6f 74 41 6c 6c 6f 77 65 64 |MethodNotAllowed|
00000080 3c 2f 6c 69 3e 0a 3c 6c 69 3e 4d 65 73      |</li>.<li>Mes|
caused by: expected element type <Error> but have <html>

Please note that, I am able to backup the data to local storage with the same cluster setup.

Dear @Munichandra Nageti, would you mind sharing this context in nebula-br repo as an issue here? https://github.com/vesoft-inc/nebula-br

Hi @wey, while using nebula-importer, i am facing below issues. can you please let me know probable cause of it?

2022/09/22 13:11:50 read /home/ec2-user/nebula-importer/examples/v2/example.yaml: is a directory

Unable to find image 'vesoft/nebula-importer:latest' locally
latest: Pulling from vesoft/nebula-importer
530afca65e2e: Pull complete 
d1122c77a8e7: Pull complete 
Digest: sha256:e28db8b80c7c35b3fc9d0a7cb71cc52fb8dc93810bc3c70a69b50bb35de429
Status: Downloaded newer image for vesoft/nebula-importer:latest
2022/09/22 13:11:50 --- START OF NEBULA IMPORTER ---
2022/09/22 13:11:50 read /home/ec2-user/nebula-importer/examples/v2/example.yaml: is a directory
2022/09/22 13:11:51 --- END OF NEBULA IMPORTER ---
[ec2-user@ip-1 v2]$ ls -ltr
total 72
-rw-rw-r-- 1 ec2-user ec2-user  60 Sep 22 13:09 student.csv
-rw-rw-r-- 1 ec2-user ec2-user  86 Sep 22 13:09 student-with-label-and-str-vid.csv
-rw-rw-r-- 1 ec2-user ec2-user  17 Sep 22 13:09 glob-follow-2.csv
-rw-rw-r-- 1 ec2-user ec2-user  51 Sep 22 13:09 glob-follow-1.csv
-rw-rw-r-- 1 ec2-user ec2-user  465 Sep 22 13:09 geography_test.csv
-rw-rw-r-- 1 ec2-user ec2-user  68 Sep 22 13:09 follow.csv
-rw-rw-r-- 1 ec2-user ec2-user  95 Sep 22 13:09 follow-with-label.csv
-rw-rw-r-- 1 ec2-user ec2-user  109 Sep 22 13:09 follow-with-label-and-str-vid.csv
-rw-rw-r-- 1 ec2-user ec2-user  114 Sep 22 13:09 follow-with-header.csv
-rw-rw-r-- 1 ec2-user ec2-user  114 Sep 22 13:09 follow-delimiter.csv
-rw-rw-r-- 1 ec2-user ec2-user 1117 Sep 22 13:09 example_with_working_dir.yaml
-rw-rw-r-- 1 ec2-user ec2-user  140 Sep 22 13:09 date_test.csv
drwxrwxr-x 2 ec2-user ec2-user  24 Sep 22 13:09 data
-rw-rw-r-- 1 ec2-user ec2-user  163 Sep 22 13:09 course.csv
-rw-rw-r-- 1 ec2-user ec2-user  152 Sep 22 13:09 course-with-header.csv
-rw-rw-r-- 1 ec2-user ec2-user  48 Sep 22 13:09 choose.csv
-rw-rw-r-- 1 ec2-user ec2-user 10416 Sep 22 13:11 example.yaml

Hi Folks, I am trying to validate the nebula snapshot operations on my cluster and as part of this I have created 3 graph spaces (

my_space_1, my_space_2 and my_space_3

) and created a snapshot. Then I deleted the namespace

my_space_3

and tried to restore the snapshot by copying/overwriting the folders

data

and

wal

in the corresponding snapshot directory to its parent directory (at the same level with checkpoints) and restarted the nebula services. But I couldn't find the deleted namespace

my_space_3

after the restart. Can someone please take a look and help me with this, Thanks.

(root@nebula) [(none)]> show spaces;
+--------------+
| Name         |
+--------------+
| "my_space_1" |
| "my_space_2" |
| "my_space_3" |
+--------------+
Got 3 rows (time spent 507/749 us)

Mon, 26 Sep 2022 10:59:24 UTC

(root@nebula) [(none)]> CREATE SNAPSHOT;
Execution succeeded (time spent 107299/107410 us)

Mon, 26 Sep 2022 10:59:29 UTC
(root@nebula) [(none)]> SHOW SNAPSHOTS;
+--------------------------------+---------+----------------------------------------+
| Name                           | Status  | Hosts                                  |
+--------------------------------+---------+----------------------------------------+
| "SNAPSHOT_2022_09_26_10_59_29" | "VALID" | "my_host_1:9779, my_host_2:9779" |
+--------------------------------+---------+----------------------------------------+
Got 1 rows (time spent 432/631 us)

(root@nebula) [(none)]> drop space my_space_3;
Execution succeeded (time spent 838/995 us)

Mon, 26 Sep 2022 11:05:40 UTC

(root@nebula) [(none)]> show spaces;
+--------------+
| Name         |
+--------------+
| "my_space_1" |
| "my_space_2" |

ubuntu@my_host:/usr/local/nebula/data/storage/nebula/1/checkpoints/SNAPSHOT_2022_09_26_10_59_29$ sudo cp -r data/ wal/ ../../
cp: 'data/OPTIONS-000007' and '../../data/OPTIONS-000007' are the same file
cp: 'data/000043.sst' and '../../data/000043.sst' are the same file
cp: 'data/000045.sst' and '../../data/000045.sst' are the same file
cp: 'data/000047.sst' and '../../data/000047.sst' are the same file
cp: 'wal/2/0000000000000030206.wal' and '../../wal/2/0000000000000030206.wal' are the same file
cp: 'wal/2/0000000000000030572.wal' and '../../wal/2/0000000000000030572.wal' are the same file
cp: 'wal/2/0000000000000000381.wal' and '../../wal/2/0000000000000000381.wal' are the same file
cp: 'wal/4/0000000000000030201.wal' and '../../wal/4/0000000000000030201.wal' are the same file

ubuntu@my_host :~$sudo /usr/local/nebula/scripts/nebula.service restart all
(root@nebula) [(none)]> show spaces;
+--------------+
| Name     |
+--------------+
| "my_space_1" |
| "my_space_2" |
+--------------+
Got 2 rows (time spent 471/657 us)

🧵 balance data support on nebula 3.2.0

Hello, is there any support/any plans for providing encryption at rest in either the OS or commercial offering? If not out of the box, does anyone know if it is doable at the RocksDB/storage level?

Hey Team, I tried to reach you guys multiple times through the website’s contact form but no response from your side. We are evaluating graph databases for one of our projects and found that Nebula can be good fit for our project. So, we would like to have technical discussion with the team to get some more insights along with enterprise pricing policies. Looking forward to hearing from you.

hi team, I has a tag with 10 milions vetex, I tried to execute nebula with cypher match use WHERE operator IN server notify error but I use Lookup on ok. thanks all

You need to have index to enable Lookup to return values without full-scan of the database, but if that ID is VertexID then it should be fine because it is implicit index. But if that thuadat.id is user field then you should create explicit index for it.

Hi @Goran Cvijanovic, thuada.id is user field, I created index for it but when I use syntax cypher MATCH then server notify high memory, can you please let know probable cause of it ? thanks you so much

Ok, will try on my dataset. I don’t use cypher with Nebula but will try. I’m on good old nebula query language, because I’m from nebula old days started using cluster in production with 1.2.1 version. Also you don’t need such large fixed-string field for UUID-s which are fixed length, use what you need for size.

I’ve tried to execute similar query on my cluster and very bad things happened. Memory is eaten on whole cluster with 8 nodes, and cluster become unusable. Some of nodes needed to be restarted because all memory, swap space and resources was exhausted and even ssh was unable to connect to the nodes.

Hi @Goran Cvijanovic and @Sơn Trần Thái Today I looked into this expression, to know this is an issue on optimizer, where the filter condition is not added in indexscan, which end up full scan of whole tag data.

(root@nebula) [basketballplayer]> explain match (v:player) where v.player.name in ["Tim Duncan"] return v limit 1
Execution succeeded (time spent 940/6819 us)

Execution Plan (optimize time 328 us)
...
-----+----------------+--------------+----------------+-----------------------------------------------
|  1 | IndexScan      | 2            |                | outputVar: {                                 |
|    |                |              |                |   "colNames": [                              |
|    |                |              |                |     "_vid"                                   |
|    |                |              |                |   ],                                         |
|    |                |              |                |   "type": "DATASET",                         |
|    |                |              |                |   "name": "__IndexScan_1"                    |
|    |                |              |                | }                                            |
|    |                |              |                | inputVar:                                    |
|    |                |              |                | space: 49                                    |
|    |                |              |                | dedup: false                                 |
|    |                |              |                | limit: 9223372036854775807                   |
|    |                |              |                | filter:                                      |
|    |                |              |                | orderBy: []                                  |
|    |                |              |                | schemaId: 50                                 |
|    |                |              |                | isEdge: false                                |
|    |                |              |                | returnCols: [                                |
|    |                |              |                |   "_vid"                                     |
|    |                |              |                | ]                                            |
|    |                |              |                | indexCtx: [                                  |
|    |                |              |                |   {                                          |
|    |                |              |                |     "columnHints": [],                       |
|    |                |              |                |     "filter": "",                            |
|    |                |              |                |     "index_id": 54                           |
|    |                |              |                |   }                                          |
|    |                |              |                | ]                                            |
-----+----------------+--------------+----------------+-----------------------------------------------
...

(root@nebula) [basketballplayer]> explain match (v:player) where v.player.name == "Tim Duncan" return v limit 1
Execution succeeded (time spent 1069/6613 us)
...
|    |                |              |                | if_track_previous_path: false            |
-----+----------------+--------------+----------------+-------------------------------------------
|  8 | IndexScan      | 2            |                | outputVar: {                             |
|    |                |              |                |   "colNames": [                          |
|    |                |              |                |     "_vid"                               |
|    |                |              |                |   ],                                     |
|    |                |              |                |   "type": "DATASET",                     |
|    |                |              |                |   "name": "__IndexScan_1"                |
|    |                |              |                | }                                        |
|    |                |              |                | inputVar:                                |
|    |                |              |                | space: 49                                |
|    |                |              |                | dedup: false                             |
|    |                |              |                | limit: 9223372036854775807               |
|    |                |              |                | filter:                                  |
|    |                |              |                | orderBy: []                              |
|    |                |              |                | schemaId: 50                             |
|    |                |              |                | isEdge: false                            |
|    |                |              |                | returnCols: [                            |
|    |                |              |                |   "_vid"                                 |
|    |                |              |                | ]                                        |
|    |                |              |                | indexCtx: [                              |
|    |                |              |                |   {                                      |
|    |                |              |                |     "columnHints": [                     |
|    |                |              |                |       {                                  |
|    |                |              |                |         "includeEnd": false,             |
|    |                |              |                |         "includeBegin": true,            |
|    |                |              |                |         "endValue": "__EMPTY__",         |
|    |                |              |                |         "beginValue": "Tim Duncan",      |
|    |                |              |                |         "scanType": "PREFIX",            |
|    |                |              |                |         "column": "name"                 |
|    |                |              |                |       }                                  |
|    |                |              |                |     ],                                   |
|    |                |              |                |     "filter": "",                        |
|    |                |              |                |     "index_id": 55                       |
|    |                |              |                |   }                                      |
|    |                |              |                | ]                                        |
-----+----------------+--------------+----------------+-------------------------------------------

The mitigation now is to use

==

instead of

IN

I had reported this https://github.com/vesoft-inc/nebula/issues/4708 Thanks!

hi @wey for your basketballplayer example, if I want where v.player.name in a list playernames then I must build v.player.name == "Tim Duncan" OR v.player.name == "Yao Ming" instead of v.player.name IN ["Tim Duncan", "Yao Ming"].

@wey If list playernames is too big that use with OR then string query can be limited characters

True, @Sơn Trần Thái, it’ll be extreme strange when we would like to do bulk conditions 😞 , we will fix this soon via https://github.com/vesoft-inc/nebula/issues/4708 .

It’s marked in v3.3.0 now and v3.3.0 will be released soon.

@wey Thanks !

hi @wey, @Goran Cvijanovic, there's a huge difference for time execution when RETURN result between two queries, can you please let know probable cause of it ? thanks

Hello everyone, How can we guarantee consistency during long calculations? For example we want to run a calculation/query on graph (database) that takes 1 minute and during the calculation another user modifies the graph (database). How can we make sure that the calculation result is valid according to state of the graph (database) when the calculation started?

I’m not sure but, if Nebula have MVCC in the core that should be a default functionality for single query request.

Hopefully a quick question? Is there support for using

named

parameters in

INSERT VERTX

statement from a Java client for instance

final String stmt = "INSERT VERTX player (name, age) VALUES 12323:($name, $age)";
client.executeWithParameter(stmt, Map.of("name", "Tony Parker", "age", 32);