This message was deleted.

for the context: one viable approach would be to use apache arrow.

Dear @Chak-Pong Chung, Welcome to the community! Nebula-Exchange is a tool to enable multiple data sources ingesting to nebulagraph, including parquet in HDFS, could this fulfill your requirements? https://docs.nebula-graph.io/3.3.0/nebula-exchange/about-exchange/ex-ug-what-is-exchange/ https://github.com/vesoft-inc/nebula-exchange

So in order to ingest parquet to nebula, we need to use the parquet reader from spark. A spark application will convert the parquet to the in-memory format Nebula needs for graph processing?

Nebula Exchange loaded them from different sources(batch or stream, parquet in HDFS is one of them, clickhouse, MySQL, hive, oracle, Kafka, Pulsar etc. are supported, too) as dataframe and then write to NebulaGraph(either via DML query or generate SST files). For now there is no direct support of Apache Arrow, while, exchange speaks JDBC, too, could we wire it like Arrow-[JDBC]-Exchange? https://docs.nebula-graph.io/3.3.0/nebula-exchange/use-exchange/ex-ug-import-from-jdbc/ If it’s not doable, could you help create an issue in nebula-exchange for adding support of apache arrow?

I am not sure whether I understand it. By JDBC, do you mean persistence of incremental update on the in-memory graph will write to a data source (like a DB) via JDBC?

Nebula has its own memory/disk format, which is not the parquet format.