This message was deleted NebulaGraph #nebula

Join Slack

This message was deleted.

# nebula

Slackbot

10/26/2022, 5:32 AM

This message was deleted.

Peicheng Yu

10/26/2022, 5:35 AM

here is the ingestion yaml in case it helps:

Copy code

version: v3
description: example graph
removeTempFiles: false
clientSettings:
  retry: 3
  concurrency: 32
  channelBufferSize: 256
  space: xxxxx (masked)
  connection:
    user: root
    password: root
    address: 127.0.0.1:9669
  postStart:
    commands: |
    afterPeriod: 8s
  preStop:
    commands: |
logPath: ./err/err_link.log
files:
  - path: <masked>
    batchSize: 256
    inOrder: true
    type: csv
    csv:
      withHeader: false
      withLabel: false
    schema:
      type: edge
      edge:
        name: communication
        withRanking: false
        srcVID:
          type: string
          index: 0
        dstVID:
          type: string
          index: 1
        props:
          - name: pic_send
            type: int
            index: 2
          - name: pic_reply
            type: int
            index: 3
          - name: message_send
            type: int
            index: 4
          - name: message_reply
            type: int
            index: 5

all the nodes have been successfully ingested in a previous run already.

Goran Cvijanovic

10/26/2022, 7:59 AM

I think that inserting on single node graphd service will be too much with 32 concurrency, I will go with 4 to be on safe side. Also do you need

inOrder: true

to be used, it is with lower impact if you have

false

for ordering.

❤️ 1

👍 1

Peicheng Yu

10/26/2022, 3:09 PM

make sense, will try that. Thanks @Goran Cvijanovic!

Peicheng Yu

10/26/2022, 6:18 PM

tried

Copy code

inOrder: false
concurrency: 2

but still getting similar errors. In addition, after the

storaged

went offline, even if I tried to start it, it doesn’t seem to work.

Copy code

peicheng.yu@peicheng-nebula-ssd-1:~$ sudo /usr/local/nebula/scripts/nebula.service start all
[WARN] The maximum files allowed to open might be too few: 1024
[ERROR] nebula-metad already running: 1920
[ERROR] nebula-graphd already running: 2016
[INFO] Starting nebula-storaged...
[INFO] Done
peicheng.yu@peicheng-nebula-ssd-1:~$ ./nebula-console -addr 127.0.0.1 -port 9669 -u root -p root

Welcome to Nebula Graph!

(root@nebula) [(none)]> SHOW HOSTS
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
| Host        | Port | HTTP port | Status    | Leader count | Leader distribution  | Partition distribution | Version |
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
| "127.0.0.1" | 9779 | 19669     | "OFFLINE" | 0            | "No valid partition" | "us_comm:1"            | "3.2.0" |
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
Got 1 rows (time spent 879/1392 us)

Not sure why the storaged refuses to be back online even if the nebula.service start command seems to be successful? Thanks.

Goran Cvijanovic

10/26/2022, 6:45 PM

I’m little bit confused here. You have only one host in your configuration, there should be at least 3 of them to have raft consensus working properly. Maybe you should go and setup your cluster using recommended configuration from documentation. Also use latest version 3.2.1 which is with some issues resolved.

Peicheng Yu

10/26/2022, 9:21 PM

Ah, make sense. Potentially the raft consensus is what’s playing a role here. Will try it with the recommended config.

Peicheng Yu

10/27/2022, 2:02 AM

A quick and naive follow up question: looks like if I want to have more than 1 host, I’ll need to append the list in

storaged.conf

Copy code

--local_ip=127.0.0.1

to be a list of IPs. But I tried

Copy code

--local_ip=127.0.0.1,127.0.0.2

Copy code

--local_ip=[127.0.0.1,127.0.0.2]

Copy code

--local_ip=127.0.0.*

but none of them seem to work. What’d be the correct way to configure it? Otherwise only one of them will be online:

Copy code

+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
| Host        | Port | HTTP port | Status    | Leader count | Leader distribution  | Partition distribution | Version |
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
| "127.0.0.1" | 9779 | 19669     | "ONLINE"  | 0            | "No valid partition" | "No valid partition"   | "3.2.0" |
| "127.0.0.2" | 9779 | 19669     | "OFFLINE" | 0            | "No valid partition" | "No valid partition"   |         |
| "127.0.0.3" | 9779 | 19669     | "OFFLINE" | 0            | "No valid partition" | "No valid partition"   |         |
+-------------+------+-----------+-----------+--------------+----------------------+------------------------+---------+
Got 3 rows (time spent 1251/1891 us)

thanks.

Peicheng Yu

10/27/2022, 3:17 AM

or is it actually not possible to have multiple hosts (i.e. high availability/reliability) in a single machine?

Goran Cvijanovic

10/27/2022, 6:46 AM

You need to have different PORT for service if you are on single machine, but it is better to use docker or other virtualisation system to have proper isolation and port mappings. Using docker (for example) will enable you to have services with same port number used, so it will be as you have normal distributed multi node cluster.

Goran Cvijanovic

10/27/2022, 6:51 AM

There is one important step to have working cluster

Starting from NebulaGraph 3.0.0, setting Storage hosts in the configuration files only registers the hosts on the Meta side, but does not add them into the cluster. You must run the
ADD HOSTS
statement to add the Storage hosts.

👍 1

Goran Cvijanovic

10/27/2022, 7:06 AM

Another thing to mention, it is actually possible to have single node Nebula version, but I didn’t try to run it.

Steps¶

Currently, you can only install standalone NebulaGraph with the source code. The steps are similar to those of the multi-process NebulaGraph. You only need to modify the step Generate Makefile with CMake by adding
-DENABLE_STANDALONE_VERSION=on
to the command. For example:

Copy code

cmake -DCMAKE_INSTALL_PREFIX=/usr/local/NebulaGraph -DENABLE_TESTING=OFF -DENABLE_STANDALONE_VERSION=on -DCMAKE_BUILD_TYPE=Release ..

Goran Cvijanovic

10/27/2022, 7:12 AM

And another info about having list of nodes for metad services, example for 10.17.0.* range

--meta_server_addrs=10.17.0.269559,10.17.0.279559,10.17.0.28:9559

so it is just comma separated list of addresses with ports included

wey

10/27/2022, 2:01 PM

Yes, as Goran said, we could configure multiple instances of service in different port to test distributed deployment(in production, it’s expected one kind of service to have singleton per OS) ~~Besides that~~ also as Goran mentioned, you could optionally leverage docker-compose to enable ease of deploy distributed cluster in single server ref: https://docs.nebula-graph.io/3.2.1/4.deployment-and-installation/2.compile-and-install-nebula-graph/3.deploy-nebula-graph-with-docker-compose/

wey

10/27/2022, 2:05 PM

Also, @Peicheng Yu,I noticed that the open file limit is too small, this will lead file access issue for storaged, too as it warned:

Copy code

peicheng.yu@peicheng-nebula-ssd-1:~$ sudo /usr/local/nebula/scripts/nebula.service start all
[WARN] The maximum files allowed to open might be too few: 1024 <------------here (wey)
[ERROR] nebula-metad already running: 1920
[ERROR] nebula-graphd already running: 2016

We need to set this limit in OS level(verify with

ulimit -a

) to enable storaged to behave normally. ref: https://docs.nebula-graph.io/3.2.1/5.configurations-and-logs/1.configurations/6.kernel-config/

👍 1

Peicheng Yu

10/27/2022, 3:20 PM

make sense, thank you both! @Goran Cvijanovic @wey Will try your recommendations above 🙂

20 Views

Open in Slack

Previous Next