Hi folks I am trying to install nebula on ec2 server from th

Question

Hi folks I am trying to install nebula on ec2 server from this <https docs nebula graph io 3 2 0 nebula operator 3 deploy nebula graph cluster 3 1create cluster with kubectl |doc> I am facing error `0 2 nodes are available 2 pod has unbound immediate PersistentVolumeClaims preemption 0 2 nodes are available 2 Preemption is not helpful for scheduling` after the installation can someone please help me

Himanshu Gupta · Answer

More inputs When I look into pvc I see they are in pending state `NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE` `metad data nebula metad 0 Pending gp2 23h` `metad log nebula metad 0 Pending gp2 23h` pvc log throws below error `waiting for a volume to be created either by external provisioner <http ebs csi aws com|ebs csi aws com> or manually created by system administrator`

wey · Answer

Hi Himanshu welcome to the community Could you plz ensure there is storage class named gp2 If not plz update corresponding sc in the crd

Himanshu Gupta · Answer

Hi wey yes there is here is the output of `kubectl get sc` ```NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE ebs sc Delete WaitForFirstConsumer false 25s gp2 default Delete Immediate false 24h```

Himanshu Gupta · Answer

```kubectl describe sc gp2 namespace=nebula operator system Name gp2 IsDefaultClass Yes Annotations Provisioner Parameters fsType=ext4 type=gp2 AllowVolumeExpansion lt unset gt MountOptions lt none gt ReclaimPolicy Delete VolumeBindingMode Immediate Events lt none gt ```

wey · Answer

< kevin qiao> could you plz help look into this

kevin.qiao · Answer

< Himanshu Gupta> Hi Himanshu please run cmd kubectl get pods l and describe the pending pod I need to confirm the reason of error preemption 0 2 nodes are available

Himanshu Gupta · Answer

```ubuntu ip ~$ kubectl get pods l namespace=nebula operator system NAME READY STATUS RESTARTS AGE nebula metad 0 0 1 Pending 0 3d19h ubuntu ip ~$ kubectl describe pod nebula metad 0 namespace=nebula operator system Name nebula metad 0 Namespace nebula operator system Priority 0 Node lt none gt Labels controller revision hash=nebula metad 7fc467d58f Annotations d6d215628f9203cb Status Pending IP IPs lt none gt Controlled By StatefulSet nebula metad Containers metad Image vesoft nebula metad v3 2 0 Ports 9559 TCP 19559 TCP 19560 TCP Host Ports 0 TCP 0 TCP 0 TCP Command bin bash ecx exec usr local nebula bin nebula metad flagfile= usr local nebula etc nebula metad conf meta server addrs=nebula metad 0 nebula metad headless nebula operator system svc cluster local 9559 local ip=$ hostname nebula metad headless nebula operator system svc cluster local ws ip=$ hostname nebula metad headless nebula operator system svc cluster local daemonize=false Limits cpu 1 memory 1Gi Requests cpu 500m memory 500Mi Readiness http get http 19559 status delay=10s timeout=5s period=10s success=1 failure=3 Environment lt none gt Mounts usr local nebula data from metad data rw path= data usr local nebula etc from nebula metad rw usr local nebula logs from metad log rw path= logs var run secrets kubernetes io serviceaccount from kube api access fvrp5 ro Conditions Type Status PodScheduled False Volumes metad log Type PersistentVolumeClaim a reference to a PersistentVolumeClaim in the same namespace ClaimName metad log nebula metad 0 ReadOnly false metad data Type PersistentVolumeClaim a reference to a PersistentVolumeClaim in the same namespace ClaimName metad data nebula metad 0 ReadOnly false nebula metad Type ConfigMap a volume populated by a ConfigMap Name nebula metad Optional false kube api access fvrp5 Type Projected a volume that contains injected data from multiple sources TokenExpirationSeconds 3607 ConfigMapName kube root ca crt ConfigMapOptional lt nil gt DownwardAPI true QoS Class Burstable Node Selectors lt none gt Tolerations op=Exists for 300s op=Exists for 300s Events Type Reason Age From Message Warning FailedScheduling 106s x1103 over 3d19h default scheduler 0 2 nodes are available 2 pod has unbound immediate PersistentVolumeClaims preemption 0 2 nodes are available 2 Preemption is not helpful for scheduling ```

Himanshu Gupta · Answer

```ubuntu ip ~$ kubectl get pvc namespace=nebula operator system NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE metad data nebula metad 0 Pending gp2 3d19h metad log nebula metad 0 Pending gp2 3d19h```

kevin.qiao · Answer

< Himanshu Gupta> please show me kubectl describe nodes outputs

Himanshu Gupta · Answer

Hi < kevin qiao> please see below ```ubuntu ip 10 0 0 0 ~$ kubectl describe nodes namespace=nebula opertor system Name ip 10 0 0 0 Roles control plane Labels Annotations unix run cri dockerd sock 0 10 0 0 0 24 192 168 1 0 true CreationTimestamp Tue 23 Aug 2022 17 57 44 +0000 Taints Unschedulable false Lease HolderIdentity ip 10 0 0 0 AcquireTime lt unset gt RenewTime Mon 29 Aug 2022 10 47 09 +0000 Conditions Type Status LastHeartbeatTime LastTransitionTime Reason Message NetworkUnavailable False Wed 24 Aug 2022 14 10 42 +0000 Wed 24 Aug 2022 14 10 42 +0000 CalicoIsUp Calico is running on this node MemoryPressure False Mon 29 Aug 2022 10 46 15 +0000 Tue 23 Aug 2022 17 57 44 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available DiskPressure False Mon 29 Aug 2022 10 46 15 +0000 Tue 23 Aug 2022 17 57 44 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure PIDPressure False Mon 29 Aug 2022 10 46 15 +0000 Tue 23 Aug 2022 17 57 44 +0000 KubeletHasSufficientPID kubelet has sufficient PID available Ready True Mon 29 Aug 2022 10 46 15 +0000 Wed 24 Aug 2022 14 05 44 +0000 KubeletReady kubelet is posting ready status AppArmor enabled Addresses InternalIP 10 0 0 0 Hostname ip 10 0 0 0 Capacity cpu 32 ephemeral storage 203070420Ki hugepages 1Gi 0 hugepages 2Mi 0 memory 261050556Ki pods 110 Allocatable cpu 32 ephemeral storage 187149698763 hugepages 1Gi 0 hugepages 2Mi 0 memory 260948156Ki pods 110 System Info Machine ID ec25c504906bf381e4fe9928fb048e97 System UUID ec25c504 906b f381 e4fe 9928fb048e97 Boot ID c87dd66c 6e90 485d a34a 0a5959eb33df Kernel Version 5 13 0 1029 aws OS Image Ubuntu 20 04 4 LTS Operating System linux Architecture amd64 Container Runtime Version Kubelet Version v1 24 4 Kube Proxy Version v1 24 4 PodCIDR 192 168 0 0 24 PodCIDRs 192 168 0 0 24 Non terminated Pods 8 in total Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age kruise system kruise daemon pcsf5 0 0 50m 0 0 0 128Mi 0 5d16h kube system calico kube controllers 5b97f5d8cf hfg7f 0 0 0 0 0 0 0 0 4d20h kube system calico node lzv9k 250m 0 0 0 0 0 0 0 4d20h kube system etcd ip 10 0 0 0 100m 0 0 0 100Mi 0 0 0 5d16h kube system kube apiserver ip 10 0 0 0 250m 0 0 0 0 0 0 0 5d16h kube system kube controller manager ip 10 0 0 0 200m 0 0 0 0 0 0 0 5d16h kube system kube proxy 6fr2q 0 0 0 0 0 0 0 0 5d16h kube system kube scheduler ip 10 0 0 0 100m 0 0 0 0 0 0 0 5d16h Allocated resources Total limits may be over 100 percent i e overcommitted Resource Requests Limits cpu 900m 2 50m 0 memory 100Mi 0 128Mi 0 ephemeral storage 0 0 0 0 hugepages 1Gi 0 0 0 0 hugepages 2Mi 0 0 0 0 Events lt none gt Name ip 10 0 1 0 Roles lt none gt Labels Annotations unix run cri dockerd sock 0 10 0 101 219 24 192 168 105 0 true CreationTimestamp Wed 24 Aug 2022 13 39 59 +0000 Taints lt none gt Unschedulable false Lease HolderIdentity ip 10 0 1 0 AcquireTime lt unset gt RenewTime Mon 29 Aug 2022 10 47 13 +0000 Conditions Type Status LastHeartbeatTime LastTransitionTime Reason Message NetworkUnavailable False Wed 24 Aug 2022 14 10 50 +0000 Wed 24 Aug 2022 14 10 50 +0000 CalicoIsUp Calico is running on this node MemoryPressure False Mon 29 Aug 2022 10 45 34 +0000 Wed 24 Aug 2022 13 39 59 +0000 KubeletHasSufficientMemory kubelet has sufficient memory available DiskPressure False Mon 29 Aug 2022 10 45 34 +0000 Wed 24 Aug 2022 13 39 59 +0000 KubeletHasNoDiskPressure kubelet has no disk pressure PIDPressure False Mon 29 Aug 2022 10 45 34 +0000 Wed 24 Aug 2022 13 39 59 +0000 KubeletHasSufficientPID kubelet has sufficient PID available Ready True Mon 29 Aug 2022 10 45 34 +0000 Wed 24 Aug 2022 14 05 32 +0000 KubeletReady kubelet is posting ready status AppArmor enabled Addresses InternalIP 10 0 0 9 Hostname ip 10 0 1 0 Capacity cpu 32 ephemeral storage 203070420Ki hugepages 1Gi 0 hugepages 2Mi 0 memory 261050556Ki pods 110 Allocatable cpu 32 ephemeral storage 187149698763 hugepages 1Gi 0 hugepages 2Mi 0 memory 260948156Ki pods 110 System Info Machine ID ec214596968bc843a7e7b569b5e3021a System UUID ec214596 968b c843 a7e7 b569b5e3021a Boot ID 7b21ffb1 b32e 4e17 a633 4f5511b6abbb Kernel Version 5 13 0 1029 aws OS Image Ubuntu 20 04 4 LTS Operating System linux Architecture amd64 Container Runtime Version Kubelet Version v1 25 0 Kube Proxy Version v1 25 0 PodCIDR 192 168 1 0 24 PodCIDRs 192 168 1 0 24 Non terminated Pods 12 in total Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age cert manager cert manager 5dd59d9d9b r9g46 0 0 0 0 0 0 0 0 5d16h cert manager cert manager cainjector 8696fc9f89 mb4c7 0 0 0 0 0 0 0 0 5d16h cert manager cert manager webhook 7d4b5b8c56 ckz2j 0 0 0 0 0 0 0 0 5d16h kruise system kruise controller manager 766b57c995 5gqhh 100m 0 200m 0 256Mi 0 512Mi 0 5d16h kruise system kruise controller manager 766b57c995 9sth9 100m 0 200m 0 256Mi 0 512Mi 0 5d16h kruise system kruise daemon zhgc4 0 0 50m 0 0 0 128Mi 0 4d21h kube system calico node ls8mf 250m 0 0 0 0 0 0 0 4d20h kube system coredns 6d4b75cb6d 2tjl7 100m 0 0 0 70Mi 0 170Mi 0 5d16h kube system coredns 6d4b75cb6d 9cmm8 100m 0 0 0 70Mi 0 170Mi 0 5d16h kube system kube proxy dkhxs 0 0 0 0 0 0 0 0 4d21h nebula operator system nebula operator controller manager deployment 5bd9fbf6b7 28jcs 200m 0 300m 0 120Mi 0 230Mi 0 4d20h nebula operator system nebula operator controller manager deployment 5bd9fbf6b7 xxgmf 200m 0 300m 0 120Mi 0 230Mi 0 4d20h Allocated resources Total limits may be over 100 percent i e overcommitted Resource Requests Limits cpu 1050m 3 1050m 3 memory 892Mi 0 1952Mi 0 ephemeral storage 0 0 0 0 hugepages 1Gi 0 0 0 0 hugepages 2Mi 0 0 0 0 Events lt none gt ```

kevin.qiao · Answer

< Himanshu Gupta> please show me kubectl get sc outputs I think metad pending was caused by storageClass gp2 gp2 is aws default storage provisioner

Himanshu Gupta · Answer

< kevin qiao> Please note that gp2 was not created automatically by aws I created it using yaml file Please let me know if you need any more details Thanks a lot for your help ```ubuntu ip ~$ kubectl get sc NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE ebs sc Delete WaitForFirstConsumer false 2d23h gp2 default Delete Immediate false 4d ubuntu ip ~$ kubectl describe sc gp2 Name gp2 IsDefaultClass Yes Annotations Provisioner Parameters fsType=ext4 type=gp2 AllowVolumeExpansion lt unset gt MountOptions lt none gt ReclaimPolicy Delete VolumeBindingMode Immediate Events lt none gt ```

kevin.qiao · Answer

< Himanshu Gupta> I see your k8s cluster was created by kubeadm I ve tested with eksctl which enable IAM role automatically I think you can check your IAM role configuration also check ebs csi controller logs please follow the link ```$ eksctl create cluster name ngtest region us east 1 ssh access ssh public key test nodes 2 vpc cidr 10 0 0 0 16 ```

Himanshu Gupta · Answer

Worked like a charm Thanks a lot < kevin qiao> adn < wey>