Deathbybandaid: /* Too many open files */

2023-04-28T10:49:53Z

Too many open files

← Older revision		Revision as of 10:49, 28 April 2023
Line 194:		Line 194:
	To remove, use the webui		To remove, use the webui
	=Troubleshooting and Error Handling Tips=		=Troubleshooting and Error Handling Tips=
	==Too many open files==		==Too many open files, OOM Killer, Out Of Memory==
	If you start receiving errors regarding "too many open files", try applying these fixes.		If you start receiving errors regarding "too many open files", try applying these fixes.

Line 209:		Line 209:
	</nowiki>		</nowiki>

	<nowiki>nano /~~etc~~/sysctl.conf</nowiki>		<nowiki>nano /proc/sys/fs/inotify/max_user_watches</nowiki>

			Make the contents of the file:
			<nowiki>1048576</nowiki>

			<nowiki>nano /usr/lib/sysctl.d/override.conf</nowiki>

	Add to the bottom of the file:		Add to the bottom of the file:
	<nowiki>		<nowiki>
Line 221:		Line 227:
	</nowiki>		</nowiki>

	~~<nowiki>nano /proc/sys/fs/inotify/max_user_watches</nowiki>~~
	~~Make the contents of the file:~~
	~~<nowiki>1048576</nowiki>~~
	Apply the New Settings:		Apply the New Settings:
	<nowiki>sysctl -p</nowiki>		<nowiki>sysctl -p</nowiki>

	<nowiki>sysctl --system</nowiki>		<nowiki>sysctl --system</nowiki>

	==increase ulimit or decrease threads==		==increase ulimit or decrease threads==
	<nowiki>lxc.prlimit.nofile = 20000</nowiki>		<nowiki>lxc.prlimit.nofile = 20000</nowiki>

Deathbybandaid: /* Add an L2 virtual switch */

2023-03-28T15:16:05Z

Add an L2 virtual switch

← Older revision		Revision as of 15:16, 28 March 2023
Line 72:		Line 72:
	==Add an L2 virtual switch==		==Add an L2 virtual switch==
	<nowiki>apt-get install openvswitch-switch</nowiki>		<nowiki>apt-get install openvswitch-switch</nowiki>

			==Add libraries for virgl==
			<nowiki>apt install libgl1 libegl1 -y</nowiki>

	==Create sudo user==		==Create sudo user==
	<nowiki>adduser sysop</nowiki>		<nowiki>adduser sysop</nowiki>

Deathbybandaid: /* Installation */

2022-03-15T16:19:26Z

Installation

@@ Line 87: / Line 87: @@
 ==Dark Theme (Optional)==
 [https://github.com/Weilbyte/PVEDiscordDark PVEDiscordDark]
 =manual add of cifs share=
   <nowiki>pvesm add cifs DrivePoolold --server 192.168.1.101 --share DrivePool --username Administrator --password ********* --smbversion 2.1</nowiki>

Deathbybandaid: Created page with " =Terminology= 'Node' is a computer running proxmox. 'Cluster' is a group of proxmox nodes, that can being used together for easy migration or high availability. 'Container' is a guest operating system that shares the kernel with the proxmox host, can give a performance boost compared to a normal Virtual OS. =VM/LXC Specifics= Proxmox VM Proxmox LXC =Installation= Download the [https://www.proxmox.com/en/downloads ISO] and install on bare metal. Alternativel..."

2021-12-08T20:19:04Z

Created page with " =Terminology= 'Node' is a computer running proxmox. 'Cluster' is a group of proxmox nodes, that can being used together for easy migration or high availability. 'Container' is a guest operating system that shares the kernel with the proxmox host, can give a performance boost compared to a normal Virtual OS. =VM/LXC Specifics= *Proxmox VM *Proxmox LXC =Installation= Download the [https://www.proxmox.com/en/downloads ISO] and install on bare metal. Alternativel..."

New page

=Terminology=
'Node' is a computer running proxmox.

'Cluster' is a group of proxmox nodes, that can being used together for easy migration or high availability.

'Container' is a guest operating system that shares the kernel with the proxmox host, can give a performance boost compared to a normal Virtual OS.
=VM/LXC Specifics=
*[[Proxmox VM]]
*[[Proxmox LXC]]
=Installation=
Download the [https://www.proxmox.com/en/downloads ISO] and install on bare metal. Alternatively, you can [https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Buster install on top of an existing Debian]
==Extra Notes==
If ZFS is the storage system of choice, it helps to install on SSD for OS, and set up a ZFS pool manually later.
==Advice on Container/VM Numbering==
Create a VLAN for your containers, and allow the container number to reflect the IP address assigned to it. This will save you a great deal of headache.

Example, say Plex, sonarr, radarr, lidarr, sabnzbd, etc are all on VLAN 12, and Plex's IP is 10.0.12.90 you can number the plex container 12090 to reflect VLANID+IP
==Enable Non-Subscription Updates==
Complete the following steps to your source lists to get updates.
===Enable No Subscription Repo===
<nowiki>nano /etc/apt/sources.list.d/pve-nosub.list</nowiki>
add:
<nowiki>deb http://download.proxmox.com/debian bullseye pve-no-subscription</nowiki>

<nowiki>nano /etc/apt/sources.list.d/pve-enterprise.list</nowiki>
===Comment out Enterprise Repo===
add a # symbol in front) of this line:
<nowiki>#deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise</nowiki>
===Install any and all available updates===
<nowiki>apt-get update && apt-get dist-upgrade -y</nowiki>
==Non-production Bleeding Edge==
===PVE Testing Repo===
<nowiki>nano /etc/apt/sources.list.d/pve-testing.list</nowiki>
add:
<nowiki>deb http://download.proxmox.com/debian/pve bullseye pvetest</nowiki>
===Debian Sid===
<nowiki>nano /etc/apt/sources.list</nowiki>
add:
<nowiki>
deb http://deb.debian.org/debian sid main non-free contrib
deb-src http://deb.debian.org/debian/ sid main contrib non-free
</nowiki>
===Install any and all available updates===
<nowiki>apt-get update && apt-get dist-upgrade -y</nowiki>
===Fix Proxmox Web Interface===
If you see:
<nowiki>unable to parse codename from '/etc/os-release' (500)</nowiki>
Then Modify the following file and add this line:
<nowiki>nano /etc/os-release</nowiki>
add:
<nowiki>VERSION_CODENAME=bullseye</nowiki>
==Disable subscription nag after login==
<nowiki>wget https://git.deathbybandaid.net/attachments/3d08b1b4-b896-497b-893b-c9c0dbb830ca</nowiki>
<nowiki>dpkg -i pve-fake-subscription_*.deb</nowiki>
<nowiki>echo "127.0.0.1 shop.maurer-it.com" | tee -a /etc/hosts</nowiki>
<nowiki>pve-fake-subscription</nowiki>
==Update Template Listing==
<nowiki>pveam update</nowiki>
==Download Virtio iso==
[https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso ISO file]
==Headless Laptop==
If you want a laptop node, you may want to disable the lid closing action, or your node will go into standby.
<nowiki>nano /etc/systemd/logind.conf</nowiki>
Change the lines below.
<nowiki>HandleLidSwitch=ignore</nowiki>
<nowiki>HandleLidSwitchDocked=ignore</nowiki>
restart the logind service.
<nowiki>systemctl restart systemd-logind.service</nowiki>
==Change Network Settings without reboot==
<nowiki>apt install ifupdown2</nowiki>
==Add an L2 virtual switch==
<nowiki>apt-get install openvswitch-switch</nowiki>
==Create sudo user==
<nowiki>adduser sysop</nowiki>

<nowiki>usermod -aG sudo sysop</nowiki>
==Install a desktop environment (optional)==
This takes a few minutes... Make sure you have a non-root user!
<nowiki>apt-get install tasksel</nowiki>

<nowiki>tasksel install desktop kde-desktop</nowiki>

<nowiki>systemctl set-default graphical.target</nowiki>

<nowiki>reboot</nowiki>
==Dark Theme (Optional)==
[https://github.com/Weilbyte/PVEDiscordDark PVEDiscordDark]
=manual add of cifs share=
<nowiki>pvesm add cifs DrivePoolold --server 192.168.1.101 --share DrivePool --username Administrator --password ********* --smbversion 2.1</nowiki>
=Clustering=
==Create a cluster==
<nowiki>pvecm create YOUR-CLUSTER-NAME</nowiki>
==Add a node to a cluster==
From the node you want to add. (new nodes must be empty)
<nowiki>pvecm add IP-ADDRESS-CLUSTER</nowiki>
==Remove a node from a cluster==
Remove all containers from the node, and login to a node that you are keeping.

it does not hurt to run these commands on all remaining nodes

Delete the node
<nowiki>pvecm delnode hp4 NODE_NAME_TO_DELETE</nowiki>
Delete node configuration files
<nowiki>cd /etc/pve/nodes</nowiki>

<nowiki>rm -r NODE_NAME_TO_DELETE</nowiki>
This step is purely to help with if you want to readd a new node with the same IP address as one you've removed
<nowiki>nano /etc/ssh/ssh_known_hosts</nowiki>
And remove ip, and node name lines
<nowiki>pvecm updatecerts</nowiki>

<nowiki>ssh-keygen -R 192.168.1.103</nowiki>

<nowiki>ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "192.168.1.103"</nowiki>
<nowiki>ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "NODE_NAME_TO_DELETE"</nowiki>
This step is when the replacement node is running:
<nowiki>ssh -o 'HostKeyAlias=pve5' root@10.0.4.5</nowiki>
==Set Node to not wait for cluster votes==
<nowiki>
pvecm expected 1
</nowiki>
=ZFS Tricks and Tips=
==Proxmox Default ZFS setup==
<nowiki>zpool create -f -o ashift=12 rpool /dev/sda /dev/sdb</nowiki>

<nowiki>zfs create rpool/data</nowiki>
==Directory mounted on ZFS==
<nowiki>zfs create rpool/downloads -o mountpoint=/downloads</nowiki>
===backups directory in ZFS===
Create Directory
<nowiki>zfs create tank/bkup -o mountpoint=/bkup</nowiki>
Add to storage.cfg
<nowiki>nano /etc/pve/storage.cfg</nowiki>

<nowiki>
dir: bkup
path /bkup
content vztmpl,iso,backup
maxfiles 4
shared 0
</nowiki>
==Add ZFS swap space==
This will add 8GB of swap space for ZFS to use.
===Verify Current===
<nowiki>root@Proxmox1:~# swapon -s</nowiki>
<nowiki>Filename Type Size Used Priority</nowiki>
<nowiki>/dev/zd0 partition 8388604 4036884 -2</nowiki>
===Add and activate===
<nowiki>root@Proxmox1:~# zfs create -V 8G rpool/swap2</nowiki>

<nowiki>mkswap /dev/zvol/rpool/swap2</nowiki>

<nowiki>swapon /dev/zvol/rpool/swap2</nowiki>
===Verify New===
<nowiki>root@Proxmox1:~# swapon -s</nowiki>
<nowiki>Filename Type Size Used Priority</nowiki>
<nowiki>/dev/zd0 partition 8388604 4036884 -2</nowiki>
<nowiki>/dev/zd96 partition 8388604 0 -3</nowiki>
===Make Load at boot===
<nowiki>nano /etc/fstab</nowiki>

<nowiki>/dev/zvol/rpool/swap2 none swap sw 0 0</nowiki>
===Rename a pool===
<nowiki>zpool export [poolname]</nowiki>
As an example, for a pool named tank which we wish to rename notankshere:
<nowiki>zpool export tank</nowiki>
Then run:
<nowiki>zpool import [poolname] [newpoolname]</nowiki>
e.g.:
<nowiki>zpool import tank notankshere</nowiki>
The pool will be imported as “notankshere” instead.
=Find and remove unused container disks=
Find:
<nowiki>pct rescan</nowiki>
Look for lines that say "add unreferenced volume"

To remove, use the webui
=Troubleshooting and Error Handling Tips=
==Too many open files==
If you start receiving errors regarding "too many open files", try applying these fixes.

These errors increase when you start running large numbers of containers.
<nowiki>nano /etc/security/limits.conf</nowiki>
Add to the bottom of the file:
<nowiki>
* soft nofile 1048576
* hard nofile 1048576
root soft nofile 1048576
root hard nofile 1048576
* soft memlock unlimited
* hard memlock unlimited
</nowiki>

<nowiki>nano /etc/sysctl.conf</nowiki>
Add to the bottom of the file:
<nowiki>
fs.inotify.max_queued_events=1048576
fs.inotify.max_user_instances=1048576
fs.inotify.max_user_watches=1048576
vm.max_map_count=262144
kernel.dmesg_restrict=1
net.core.rmem_max = 16777216
fs.file-max = 1048576
</nowiki>

<nowiki>nano /proc/sys/fs/inotify/max_user_watches</nowiki>
Make the contents of the file:
<nowiki>1048576</nowiki>
Apply the New Settings:
<nowiki>sysctl -p</nowiki>

<nowiki>sysctl --system</nowiki>
==increase ulimit or decrease threads==
<nowiki>lxc.prlimit.nofile = 20000</nowiki>
==Container Names missing in left pane==
<nowiki>
service pvedaemon restart
</nowiki>

<nowiki>
service pvestatd restart
</nowiki>

<nowiki>
service pveproxy restart
</nowiki>
==Backup Fails because ZFS Snapshot is already there==
I got an error on backup that said:
<nowiki>
INFO: starting new backup job: vzdump 190 --remove 0 --compress lzo --node Proxmox1 --storage DBB-Proxmox --mode snapshot
INFO: filesystem type on dumpdir is 'cifs' -using /var/tmp/vzdumptmp29423 for temporary files
INFO: Starting Backup of VM 190 (lxc)
INFO: status = running
INFO: CT Name: sonarr.dbb.local
INFO: excluding bind mount point mp0 ('/Drivepool') from backup
INFO: excluding bind mount point mp1 ('/Downloads') from backup
INFO: backup mode: snapshot
INFO: ionice priority: 7
INFO: create storage snapshot 'vzdump'
snapshot create failed: starting cleanup
no lock found trying to remove 'backup' lock
ERROR: Backup of VM 190 failed - zfs error: cannot create snapshot 'rpool/data/subvol-190-disk-1@vzdump': dataset already exists
INFO: Backup job finished with errors
TASK ERROR: job errors
</nowiki>
The solution was to run:
<nowiki>
root@Proxmox1:~# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
rpool/data/subvol-190-disk-1@vzdump 86.5M - 2.13G -
root@Proxmox1:~# zfs destroy rpool/data/subvol-190-disk-1@vzdump
root@Proxmox1:~# zfs list -t snapshot
no datasets available
</nowiki>
==Found stale copy during migration==
I got this message
<nowiki>
ERROR: found stale volume copy 'local-zfs:subvol-2032-disk-0' on node 'Proxmox-HP'
</nowiki>
Solution (on host you are migrating TO):
<nowiki>
qm rescan --vmid 2032

OR for LXC

pct rescan --vmid 2032
</nowiki>

<nowiki>
zfs list -rt all rpool

zfs destroy rpool/data/subvol-2032-disk-0

</nowiki>

Proxmox-VE - Revision history

Deathbybandaid: /* Too many open files */

Deathbybandaid: /* Add an L2 virtual switch */

Deathbybandaid: /* Installation */