nebulagraph #nebula

A quick question regarding using nebula-importer: I’m installing Nebula on a GCP compute instance, and trying to ingest data from csv files locally in the compute instance. The node ingest is usually pretty smooth, but when doing the edge ingestion, very often I’d incur errors like

Copy code

...
2022/10/25 23:06:06 [INFO] statsmgr.go:89: Tick: Time(90.00s), Finished(8826624), Failed(0), Read Failed(0), Latency AVG(74173us), Batches Req AVG(74755us), Rows AVG(98072.47/s)
2022/10/25 23:06:11 [INFO] statsmgr.go:89: Tick: Time(95.00s), Finished(8826624), Failed(0), Read Failed(0), Latency AVG(74173us), Batches Req AVG(74755us), Rows AVG(92911.68/s)
2022/10/25 23:06:13 [ERROR] handler.go:63: Client 8 fail to execute: INSERT EDGE `communication`(`pic_send`,`pic_reply`,`message_send`,`message_reply`) VALUES  "cd175378-5b57-40c0-8044-abcefdcccxxx"->"f9aad3da-8fe7-41d2-be7e-9866xxx13e9e":(0,0,23,15) , "f06b5211-bf19-4dxxxx78f-e32da48c390e"->"d038xxxx-4cbd-47ca-869c-91571b4697f0":(0,0,63,51) , ...

and when I check the local storage service, it was offline.

Copy code

+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
| Host        | Port | HTTP port | Status    | Leader count | Leader distribution  | Partition distribution | Version |
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
| "127.0.0.1" | 9779 | 19669     | "OFFLINE" | 0            | "No valid partition" | "us_comm:1"            | "3.2.0" |
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
Got 1 rows (time spent 1303/1945 us)

although I checked right before the ingestion to make sure

nebula-storaged

was online. From the logging above, does it mean

storaged

was offline roughly 95 seconds after the ingestion started? I’ve encountered this several times, but not sure what the root cause yet. If anyone can shed some light, I’d really appreciate. Thanks.

Rama Raju

10/27/2022, 10:37 AM

👋 Hi everyone!

👋 1

wey

10/27/2022, 1:58 PM

Rama! Welcome to the NebulaGraph Community 🙂

Hera Han

11/01/2022, 9:28 AM

Hi guys, NebulaGraph Community Meeting will be held tomorrow 🎉 Staff Engineer @vesoft, Dr.Chen will share the topic: Refactoring and data structure evolution of NebulaGraph Binding Table. Welcome to find out more and join us:https://www.meetup.com/nebulagraph/events/288943582/

❤️ 2

Zhong TIAN

11/02/2022, 7:46 AM

Hi, Guys, join our monthly community meetup today at 1330 UTC to learn about new progress at #NebulaGraph, an #opensource distributed #graphdatabase. Also, enjoy our native developer, Dr. Xuntao CHENG, sharing his recent work on NebulaGraph BindingTable. https://us02web.zoom.us/meeting/register/tZ0rcuypqDMvGdLuIm4VprTlx96wrEf062SH

Will Droste

11/03/2022, 2:20 PM

Is there a recording? ^^

Jeremy Simpson

11/09/2022, 7:55 PM

I was wondering what scheduler pods did in nebula-operator. There doesn’t seem to be any documentation on it that I could find

❤️ 1

Will Droste

11/13/2022, 4:14 PM

I was looking at the Java client and I was wondering if this is true from the server perspective, it appears there’s a 1-to-1 between connection and server session. I know that sessions though remain even after all connections are closed. So can you just pass in the

sessoinId

in to any connection? Is the session ID unique enough not be a security concern. For instance if you make a connection to Nebula can somebody guess a

sessionId

just be doing

YIELD 1

statements?

❤️ 1

Will Droste

11/13/2022, 4:15 PM

Also though from a resource perspective it seems then I can re-connect and just keep using the same session ID, which is good considering that the default expiration for a session is 8hrs. From a security perspective though if I can reuse the session ID and not have to auth then there should be additional protections in place like this session came from this IP addresss.

Will Droste

11/15/2022, 9:42 PM

For Nebula 3.0.2+ i see support for ARM on the nebula-cluster and that's great because I have team members w/ M1 macs however when you try the

docker-compose

to build a cluster locally the

nebula-console

is still amd64 and also its says it fails to connect

Will Droste

11/15/2022, 9:43 PM

Here's the output

Will Droste

11/15/2022, 9:43 PM

Copy code

[+] Running 11/11
 ⠿ Network v3_3_nebula-net     Created                                                                                                        0.0s
 ⠿ Container v3_3-metad2-1     Created                                                                                                        0.1s
 ⠿ Container v3_3-metad1-1     Created                                                                                                        0.1s
 ⠿ Container v3_3-metad0-1     Created                                                                                                        0.1s
 ⠿ Container v3_3-storaged2-1  Created                                                                                                        0.1s
 ⠿ Container v3_3-storaged1-1  Created                                                                                                        0.1s
 ⠿ Container v3_3-storaged0-1  Created                                                                                                        0.1s
 ⠿ Container v3_3-graphd1-1    Created                                                                                                        0.1s
 ⠿ Container v3_3-graphd-1     Created                                                                                                        0.1s
 ⠿ Container v3_3-graphd2-1    Created                                                                                                        0.1s
 ⠿ Container v3_3-console-1    Created                                                                                                        0.0s
Attaching to v3_3-console-1, v3_3-graphd-1, v3_3-graphd1-1, v3_3-graphd2-1, v3_3-metad0-1, v3_3-metad1-1, v3_3-metad2-1, v3_3-storaged0-1, v3_3-storaged1-1, v3_3-storaged2-1
v3_3-console-1    | 2022/11/15 21:33:58 Fail to initialize the connection pool, host: graphd, port: 9669, failed to open connection, error: failed to open transport, error: dial tcp 172.22.0.10:9669: connect: connection refused 
v3_3-console-1    | panic: Fail to initialize the connection pool, host: graphd, port: 9669, failed to open connection, error: failed to open transport, error: dial tcp 172.22.0.10:9669: connect: connection refused 
v3_3-console-1    | 
v3_3-console-1    | goroutine 1 [running]:
v3_3-console-1    | log.Panicf(0xc00017a0c0, 0xbc, 0x0, 0x0, 0x0)
v3_3-console-1    | 	/usr/local/go/src/log/log.go:345 +0xc0
v3_3-console-1    | main.main()
v3_3-console-1    | 	/usr/src/main.go:549 +0x8d0
v3_3-console-1 exited with code 2

Will Droste

11/15/2022, 9:44 PM

https://github.com/vesoft-inc/nebula-docker-compose

H3cT0R

11/19/2022, 4:11 PM

Hi I'm trying to use cpp client, via the Session Connection example. But hit a blocker. Is there any documentation or wiki, which I can refer to.

Chak-Pong Chung

11/21/2022, 1:11 AM

Hi! what does nebula use to import property data of parquet format into memory?

wey

11/23/2022, 10:57 AM

Dear all We will have an ad-hoc meeting with Associate Professor Salihoğlu from UWaterloo to share the new OpenSource Graph DBMS: Kùzu • https://t.co/1KwutXlWti • passcode: 105729 Title: Kùzu Graph Database Management System Abstract: In this talk, I will present the Kùzu graph database management system (GDBMS): an embeddable and feature-rich open-source DBMS that is optimized for ease of use, performance, and scalability. Datasets and workloads of popular applications that use GDBMSs require a set of storage and query processing features that relational DBMSs (RDBMSs) do not traditionally optimize for. These include optimizations for: (i) many-to-many (m-n) joins; (ii) cyclic joins; (iii) recursive joins; (iv) semi-structured data storage; and (v) support for universal resource identifiers. Kùzu aims to integrate state-of-art storage, indexing, and query processing techniques to highly optimize for this feature set. I will start by presenting the overall vision of Kùzu and then talk about the novel join operators in the system that performs joins using compressed factorized relation representations. In its current usage vision, Kùzu aims to be the go-to DBMS for building graph data science pipelines. Kùzu is actively being developed as an embeddable and fully functional open-source DBMS and is released under a permissible license. Bio: Semih Salihoğlu is an Associate Professor and a David R. Cheriton Faculty Fellow at University of Waterloo. His research focuses on developing systems for managing, querying, or doing analytics on graph-structured data. His main on-going systems project is Kùzu, which is a new graph database management system that integrates novel storage, indexing and query processing techniques. He holds a PhD from Stanford University and is a recipient of the VLDB 2018 Best Paper and the VLDB 2022 Best Experiments and Analysis Paper awards.

Dominik

11/24/2022, 4:45 PM

Hi everyone. I ran into a problem starting nebula graph. metad and storaged always exit immediately due to some errors.

Will Droste

11/28/2022, 3:43 PM

@wey is there's a road map for getting this working

Will Droste

11/28/2022, 3:43 PM

Copy code

It is not yet supported to add property conditions when matching vertices with multiple tags.

Will Droste

12/08/2022, 8:42 PM

Got question is there any plan to support byte[] data in nebula? From the graph? The thrift api supports and I was thinking of encrypted values.

porsche

12/13/2022, 5:54 AM

Has anyone tried to set Nebula option enable_partitioned_index_filter and max_edge_returned_per_vertex?

porsche

12/13/2022, 5:55 AM

Using Nebula helm charts?

Will Droste

12/14/2022, 6:58 AM

@wey with the absence of a unique constraint or index.. is there a good

merge

type query that can try to reduce the possibly of duplicates.. where Nebula does a search before an insert.

porsche

12/27/2022, 1:53 AM

Did anyone try log clean up using Cronjob? Can I get code snippet please?

Chak-Pong Chung

01/07/2023, 12:06 AM

hello! anyone here tried using ldbc to benchmark nebula?

Chiranjeevi

02/02/2023, 5:59 AM

Hi I have installed nebulagraph standalone for linux by building from source. Graph service is running in 127.0.0.1:9669. However I'm not able to make the nebula client work. I'm trying the exmaple code SessionExample.cpp but the execution gets stuck in ConnectionPool init function and doesnt proceed after that. Aybody know how to fix this issue? Thank you

Alex Kumundzhiev

02/15/2023, 1:00 PM

Hello, do you have estimations for maximum data size with Nebula? Such as N TB graph tested, number of V, E

Sơn Trần Thái

02/21/2023, 8:52 AM

Hi everyone, Why The

WHERE

clause cannot be used in an

OPTIONAL MATCH

clause in version 3.4 ?

Duyên Đỗ (D Duck)

03/06/2023, 3:16 AM

I want to rebuild the new index, but when the rebuild fails, it is recorded in the logs. Hope everyone supports E20230306 100249.626945 2099 StorageAccessExecutor.h:39] InsertEdgesExecutor failed, error E_EDGE_NOT_FOUND, part 2 E20230306 100249.627936 2094 QueryInstance.cpp:137] Storage Error: Edge not found. E20230306 100313.974483 2094 StorageAccessExecutor.h:39] InsertEdgesExecutor failed, error E_EDGE_NOT_FOUND, part 12 E20230306 100313.975497 2094 QueryInstance.cpp:137] Storage Error: Edge not found. E20230306 100313.987063 2113 StorageAccessExecutor.h:39] InsertEdgesExecutor failed, error E_EDGE_NOT_FOUND, part 12 E20230306 100313.987349 2109 QueryInstance.cpp:137] Storage Error: Edge not found. E20230306 100314.006799 2099 StorageAccessExecutor.h:39] InsertEdgesExecutor failed, error E_EDGE_NOT_FOUND, part 2 E20230306 100314.007014 2099 QueryInstance.cpp:137] Storage Error: Edge not found. E20230306 100315.773773 2103 StorageAccessExecutor.h:39] InsertEdgesExecutor failed, error E_EDGE_NOT_FOUND, part 2 E20230306 100315.774618 2103 QueryInstance.cpp:137] Storage Error: Edge not found.