• ProxMox, NAS & backups - oh my.

    From paulie420@21:2/150 to All on Monday, July 04, 2022 18:34:01
    Some of you know that I recently moved 2o bbS over to the ProxMox server - and that has been wonderful. Its running great, got all the little bugs ironed out [mostly!] and am enjoying both the .qcow format that the VM is saved in, and PVE [ProxMox] backups that run easily and are easy to handle.

    I also have a NAS, still running on a Raspberry Pi, that will shortly find ITS way over to the ProxMox host... when I created the NAS I was struggling a bit with understanding permissions and... does openmediavault need a USER account to give permissions to the bbS box to read/write to the NFS share, and... just exactly grasping how it w0rks. Got it setup - but wonder if I could have tighter settings, or do better with fully understanding how those permissions work...

    Today I find ProxMox Backup Server. It looks GREAT, and it can do automated things much better than simply using the backups tab in PVE... so I'm excited to use it as a solution to do all backups for my various PVE VMs. However, PBS [ProxMox Backup Server] prefers you to have a physical drive for those backups... since its running ON the PVE, thats not super easy to do - while I could give one of my server drives to it, I don't really want to make that change... I don't think. It will accept NAS/NFS mounts as a DataShare, but I'm again running into permissions issues. PBS won't let me mount the thing - even after I created a SEPARATE NFS share and made sure it had read/write perms... I know I'm just not understanding things...

    Just venting, not asking for explanations - I would accept any 'newbie' websites that people might suggest; but I get the best information directly from the services above's documentation. It certainly could be easier, but I hope in the end I can create a setup that is secure and ISN'T just set to wide open... the controls within all of these systems is great; you just have to be a knowledgable sysOp. :P Thats the hard bit.

    Having fun in techno-land... still learning how to string em' all together.



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 2022/04/26 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)
  • From 2twisty@21:3/166 to paulie420 on Tuesday, July 05, 2022 10:07:08
    PVE VMs. However, PBS [ProxMox Backup Server] prefers you to have a physical drive for those backups... since its running ON the PVE, thats

    Maybe I'm misreading your post... But it sounds like you are trying to put your backup storage in the same physical box as the VMs you want to back up.

    That's a really bad idea. If you have a hardware failure, you could lose both the VMs *and* the backups in one fell swoop.

    I strongly recommend that you keep your backup target on separate hardware -- leave it on the Pi if your backups don't need much bandwidth on the network. Build a separate box if needed.

    I am not familiar with OpenMediaVault's NFS config parameters, so I can't give specific advice there. I use TrueNAS to host the storage of my VMs over NFS. I then use ZFS replication to perform the backups. This is one thing I plan to change in the future, but it's working well for now.

    I am also not familiar with the way Proxmox does its connections. So this could be a misconfig on either end. Are you using NFS 3 or NFS 4? NFS 4 allows for more security, but in my little private homelab, I run NFS 3 for simplicity.

    Sorry that I don't have more constructive advice. I just wanted to jump in and advise to NOT put your backups on the same hardware as the stuff you are backing up.

    Hope this helps (some),

    Chad

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: The Ratrace Losers (21:3/166)
  • From claw@21:1/210 to paulie420 on Tuesday, July 05, 2022 18:18:25
    On 04 Jul 2022, paulie420 said the following...

    Some of you know that I recently moved 2o bbS over to the ProxMox server
    - and that has been wonderful. Its running great, got all the little
    bugs ironed out [mostly!] and am enjoying both the .qcow format that the VM is saved in, and PVE [ProxMox] backups that run easily and are easy
    to handle.

    I also have a NAS, still running on a Raspberry Pi, that will shortly
    find ITS way over to the ProxMox host... when I created the NAS I was struggling a bit with understanding permissions and... does
    openmediavault need a USER account to give permissions to the bbS box to read/write to the NFS share, and... just exactly grasping how it w0rks. Got it setup - but wonder if I could have tighter settings, or do better with fully understanding how those permissions work...

    Today I find ProxMox Backup Server. It looks GREAT, and it can do automated things much better than simply using the backups tab in PVE... so I'm excited to use it as a solution to do all backups for my various PVE VMs. However, PBS [ProxMox Backup Server] prefers you to have a physical drive for those backups... since its running ON the PVE, thats not super easy to do - while I could give one of my server drives to it,
    I don't really want to make that change... I don't think. It will accept NAS/NFS mounts as a DataShare, but I'm again running into permissions issues. PBS won't let me mount the thing - even after I created a
    SEPARATE NFS share and made sure it had read/write perms... I know I'm just not understanding things...

    Just venting, not asking for explanations - I would accept any 'newbie' websites that people might suggest; but I get the best information directly from the services above's documentation. It certainly could be easier, but I hope in the end I can create a setup that is secure and ISN'T just set to wide open... the controls within all of these systems
    is great; you just have to be a knowledgable sysOp. :P Thats the hard
    bit.

    Having fun in techno-land... still learning how to string em' all together.

    pAULIE42o

    Well I've not used proxmox before but I do know on ESXi (VMWare) it likes iSCSI. Basically this will act as an attach SCSI drive but can be across the network. the one real downside it only one server should ever use it at a time. Not to be confused with VMs. So the drive would be dedicated to the server thats using it. Keep in mind as many VMs can access this through that server as you like. I have hear of people being successful with using it with more that one device typically with PXE booting but then some of them might be in read only mode. Not worth the risk though. iSCSI will require a whole nother computer to host it a PI might do it but I have not worked with it on a PI. My experience was with Ubuntu server. There is a bit of a learning curve but iSCSI is fast and works with most things. Hopefully a proxmox guru can step in and verify it this would be a good route.

    As always if I can be of any assistance let me know.

    |23|04Dr|16|12Claw
    |16|14Sysop |12Noverdu |14BBS |04(|14Noverdu.com|04)
    |10Standard Ports for SSH/Telnet Web/HTTP://|14Noverdu.com:808
    |20|15fsxNet/MRC Chat/Registered Doors!/50Nodes/No Time Use! Stay On!|16|07

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: Noverdu BBS (21:1/210)
  • From paulie420@21:2/150 to 2twisty on Tuesday, July 05, 2022 19:43:52
    PVE VMs. However, PBS [ProxMox Backup Server] prefers you to have a physical drive for those backups... since its running ON the PVE, tha

    Maybe I'm misreading your post... But it sounds like you are trying to put your backup storage in the same physical box as the VMs you want to back up.

    That's a really bad idea. If you have a hardware failure, you could
    lose both the VMs *and* the backups in one fell swoop.

    So I was going to mount an external NFS to /mnt/somewhere and backup that way, but have since learned that its not suggested or supported by ProxMox Backup Server... I now know.

    I [like to] use Raspberry Pi's for simple server stuff, and theres no Proxmox Backup Server for ARM, so... I'm thinking about how I want to tackle backups.

    Currently I'm just using PVEs Backups tab to backup ProxMox VMs, and thinking about setting up a Raspberry Pi w/ NFS drives attached to go in and pull out those PVE backups.... I Guess I should plan on using some x86 machine to run PBS and get it running a bit better - but me thinking I'm the king Linux sysOp and all, I think I can do it myself.

    I understand some of the benefits of using PBS and hopefully will get there one day... thanks for the response, as I wasn't aware that I was attempting to set it up backwards. :P Appreciated.



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 2022/04/26 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)
  • From paulie420@21:2/150 to claw on Tuesday, July 05, 2022 19:47:02
    Well I've not used proxmox before but I do know on ESXi (VMWare) it likes iSCSI. Basically this will act as an attach SCSI drive but can be
    across the network. the one real downside it only one server should
    ever use it at a time. Not to be confused with VMs. So the drive would be dedicated to the server thats using it. Keep in mind as many VMs can access this through that server as you like. I have hear of people
    being successful with using it with more that one device typically with PXE booting but then some of them might be in read only mode. Not worth the risk though. iSCSI will require a whole nother computer to host it
    a PI might do it but I have not worked with it on a PI. My experience
    was with Ubuntu server. There is a bit of a learning curve but iSCSI is fast and works with most things. Hopefully a proxmox guru can step in
    and verify it this would be a good route.

    As always if I can be of any assistance let me know.

    Thanks for your info - my issue is that... I have an x86 server and wanted to use it to RUN the backup server; since have learned reasons why this isn't a great idea... but I don't have several x86 machines sitting around - well, I guess I do if I used ThinkPads; but I don't really wanna go down that half-assed road... I have a plethora of Raspberry Pi's, but PBS [ProxMox Backup Server] has no ARM arm... :P So at the moment, I'm just building a basic Debian Raspberry Pi instance to go in and suck out the PVE [ProxMox] backups and pull those out to an external drive connected to said RPi.

    It isn't perfect, but hopefully will tide me over until I can dedicate another x86 machine to host PBS.

    Its fun being a shoestring sysOp. w00t w00t.



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 2022/04/26 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)
  • From claw@21:1/210 to paulie420 on Wednesday, July 06, 2022 08:01:11
    On 05 Jul 2022, paulie420 said the following...
    Raspberry Pi's, but PBS [ProxMox Backup Server] has no ARM arm... :P So at the moment, I'm just building a basic Debian Raspberry Pi instance to go in and suck out the PVE [ProxMox] backups and pull those out to an external drive connected to said RPi.
    pAULIE42o

    It doesn't matter if proxmox has an arm edition. If you setup iSCSI and connect it to that system it will look like its just another connected drive to the system. A little Googleing showed me that its very doable. On your pie you set up the drive you want as an iSCSI and then on the proxmox system your connect the iSCSI drive. The system will treat it like any other connected drive after that. You can format partition so on and so forth. Literally like it any other connected drive.

    Its not easy at first but its worth it. I backed up my whole Server this way when I needed to transition it to new hardware. I also have used it for VM storage.

    |23|04Dr|16|12Claw
    |16|14Sysop |12Noverdu |14BBS |04(|14Noverdu.com|04)
    |10Standard Ports for SSH/Telnet Web/HTTP://|14Noverdu.com:808
    |20|15fsxNet/MRC Chat/Registered Doors!/50Nodes/No Time Use! Stay On!|16|07

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: Noverdu BBS (21:1/210)
  • From 2twisty@21:3/166 to claw on Wednesday, July 06, 2022 12:22:05
    Well I've not used proxmox before but I do know on ESXi (VMWare) it likes iSCSI. Basically this will act as an attach SCSI drive but can be

    I use NFS for my storage instead of iSCSI, since TrueNAS's NFS performance is a smidge better than its iSCSI performance. I have used both. The advantage to NFS is that your .vmdk files are not locked up in a VMFS volume, and are just regular files on the NFS, making backups and transplants a bit simpler.

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: The Ratrace Losers (21:3/166)
  • From 2twisty@21:3/166 to paulie420 on Wednesday, July 06, 2022 12:27:58
    why this isn't a great idea... but I don't have several x86 machines sitting around - well, I guess I do if I used ThinkPads; but I don't

    Just about any old beater PC from the thrift store should give you what you need.

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: The Ratrace Losers (21:3/166)
  • From paulie420@21:2/150 to claw on Wednesday, July 06, 2022 17:58:35
    It doesn't matter if proxmox has an arm edition. If you setup iSCSI and connect it to that system it will look like its just another connected drive to the system. A little Googleing showed me that its very doable. On your pie you set up the drive you want as an iSCSI and then on the proxmox system your connect the iSCSI drive. The system will treat it like any other connected drive after that. You can format partition so
    on and so forth. Literally like it any other connected drive.

    Its not easy at first but its worth it. I backed up my whole Server
    this way when I needed to transition it to new hardware. I also have
    used it for VM storage.

    Oh I understand what yer suggesting to do - instead of simply using my NAS/NFS drives [which, btw, PBS says CAN be done - but they don't support it...] I could setup some Pi w/ iSCSI and use that to mount to the PBS VM that live within PVE.... ok.

    I'll look into that, as currently I was just gonna setup a Pi to pull the PVE backups to. I haven't nuked the PBS VM yet, so - will do some reading. Thanks for the suggestion.



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 2022/04/26 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)
  • From paulie420@21:2/150 to 2twisty on Wednesday, July 06, 2022 17:59:55
    Just about any old beater PC from the thrift store should give you what you need.

    You know, I didn't think about that - I do have an old mid-tower thats only 6-7 years old... I'm sure it'd be enough - but I think I stipped the old HDD out of it since the gal didn't want her data to float around...

    Hmmmm - maybe that would be a decent solution.



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 2022/04/26 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)
  • From claw@21:1/210 to paulie420 on Thursday, July 07, 2022 07:56:14
    On 06 Jul 2022, paulie420 said the following...
    Oh I understand what yer suggesting to do - instead of simply using my NAS/NFS drives [which, btw, PBS says CAN be done - but they don't
    support it...] I could setup some Pi w/ iSCSI and use that to mount to
    the PBS VM that live within PVE.... ok.

    I'll look into that, as currently I was just gonna setup a Pi to pull
    the PVE backups to. I haven't nuked the PBS VM yet, so - will do some reading. Thanks for the suggestion.



    pAULIE42o

    Close. Instead of mounting it with the VM mount it on the host machine. This way its available to all VMs and you should be able to just use the back up to sent it there just like it was a physically connected drive. But I get it if your already set on NFS. Then you have your solution. I've not used TrueNAS on a PI but hay if you get it going I would be interested in checking that out. TrueNAS is cool.

    |23|04Dr|16|12Claw
    |16|14Sysop |12Noverdu |14BBS |04(|14Noverdu.com|04)
    |10Standard Ports for SSH/Telnet Web/HTTP://|14Noverdu.com:808
    |20|15fsxNet/MRC Chat/Registered Doors!/50Nodes/No Time Use! Stay On!|16|07

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: Noverdu BBS (21:1/210)
  • From poindexter FORTRAN@21:4/122 to paulie420 on Wednesday, July 06, 2022 08:58:00
    paulie420 wrote to 2twisty <=-

    Currently I'm just using PVEs Backups tab to backup ProxMox VMs

    Ditto here. I have a Synology that's mounted via NFS to Proxmox and use the built-in backup utility. I'm going to set up cloud sync with Synology to get
    a backup offsite, and figure some bare-metal backup for Proxmox, probably
    just a cpio job to the Synology.

    Another cheap and dirty option with Proxmox VE would be to run NFS or Samba from a router - I used a Netgear router with DD-WRT to mount a drive and use it as a backup target.
    --- SBBSecho 3.15-Win32
    * Origin: realitycheckBBS.org -- information is power. (21:4/122)
  • From vorlon@21:1/195.1 to paulie420 on Friday, July 08, 2022 11:41:27
    Hi Paulie420,

    Oh I understand what yer suggesting to do - instead of simply using my NAS/NFS drives [which, btw, PBS says CAN be done - but they don't
    support it...] I could setup some Pi w/ iSCSI and use that to mount to
    the PBS VM that live within PVE.... ok.

    I'll look into that, as currently I was just gonna setup a Pi to pull
    the PVE backups to. I haven't nuked the PBS VM yet, so - will do some reading. Thanks for the suggestion.

    The proxmox backup your using from it's web interface uses what's called
    vzdump to make a backup. It can be scripted to do so much more.

    I have one of mine doing a remote nfs backup a couple of times a day.
    Along with a local dump to a spare drive in the server at different times.
    The crontab event just runs different scripts at different times.


    crontab:
    0 4 * * * root /root/scripts/lnx.sh
    0 8 * * * root /root/scripts/lnx.sh


    fstab on proxmox server:

    # remote system with nfs share
    192.168.1.254:/var/nfsroot/syd /mnt/syd nfs rw,async,hard,intr,noexec 0 0

    The the nfs backup script:

    #/bin/bash
    #
    # copy a new nfs vzdump config
    cp /root/scripts/vzdump.syd /etc/vzdump.conf
    mount /mnt/syd
    vzdump 100
    vzdump 101
    vzdump 104
    umount /mnt/syd


    vzdump.conf:

    tmpdir: /bck/tmp
    dumpdir: /mnt/syd
    mode: snapshot
    keep-last: 20
    remove: 1
    compress: gzip
    mailto: fred@domain.com







    \/orlon
    aka
    Stephen


    --- Talisman v0.42-dev (Linux/m68k)
    * Origin: Vorlon Empire: Amiga 3000 powered in Sector 550 (21:1/195.1)
  • From 2twisty@21:3/166 to claw on Friday, July 08, 2022 10:57:15
    I've not used TrueNAS on a PI but hay if you get it going I would be interested in checking that out. TrueNAS is cool.

    Sadly, TrueNAS is x86 only at this time -- If they do make an ARM compile, I doubt it will run well (if at all) on a Pi because of the lower spec hardware.

    TrueNAS's main function is to support ZFS, and to do that well requires direct access to hard drive controllers and lots of RAM, neither of which a Pi has.

    Jeff Geerling will probably try it with a Compute Module 4 if TrueNAS ever comes out in an ARM version, but many have already tried to set up ZFS on a Pi with very slow (and sometimes unreliable) results.

    If you are just looking for NAS functions (NFS, iSCSI, SMB, etc) then OpenMediaVault is a good choice on a Pi..

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: The Ratrace Losers (21:3/166)
  • From paulie420@21:2/150 to vorlon on Friday, July 08, 2022 13:18:04
    The proxmox backup your using from it's web interface uses what's called vzdump to make a backup. It can be scripted to do so much more.

    I have one of mine doing a remote nfs backup a couple of times a day. Along with a local dump to a spare drive in the server at different
    times. The crontab event just runs different scripts at different times.

    Ahhh, thanks for reminding my about the fact that all PVE tools have CLI commands to do all the things most ppl use the web interface for - I was going to use a RPi backup server machine to do the backups, but yea - I could just tell PVE to send them over... awesome.

    Maybe using PVE .vma.gz backups will be enough for me - I don't know that I *need* an entire PBS system.

    crontab:
    0 4 * * * root /root/scripts/lnx.sh
    0 8 * * * root /root/scripts/lnx.sh


    fstab on proxmox server:

    # remote system with nfs share
    192.168.1.254:/var/nfsroot/syd /mnt/syd nfs rw,async,hard,intr,noexec 0 0

    The the nfs backup script:

    #/bin/bash
    #
    # copy a new nfs vzdump config
    cp /root/scripts/vzdump.syd /etc/vzdump.conf
    mount /mnt/syd
    vzdump 100
    vzdump 101
    vzdump 104
    umount /mnt/syd

    vzdump.conf:

    tmpdir: /bck/tmp
    dumpdir: /mnt/syd
    mode: snapshot
    keep-last: 20
    remove: 1
    compress: gzip
    mailto: fred@domain.com


    Ok - now this is just overkill!!! Thanks a ton for all the data here, I'll be sure to extract this message and use it to help setup my solution. GREAT STUFF - thanks, vorlon!



    |07p|15AULIE|1142|07o
    |08.........

    --- Mystic BBS v1.12 A48 2022/04/26 (Linux/64)
    * Origin: 2o fOr beeRS bbs>>>20ForBeers.com:1337 (21:2/150)
  • From vorlon@21:1/195.1 to 2twisty on Saturday, July 09, 2022 11:33:25
    Hi 2twisty,

    TrueNAS's main function is to support ZFS, and to do that well
    requires direct access to hard drive controllers and lots of RAM,
    neither of which a Pi has.

    The Pie4 8G would help in the ram side, but the slow disk access via USB
    would be a reall killer to performance.

    That's one thing that would make a pi so much better, is if they had two
    sata ports.


    \/orlon
    aka
    Stephen


    --- Talisman v0.42-dev (Linux/m68k)
    * Origin: Vorlon Empire: Amiga 3000 powered in Sector 550 (21:1/195.1)
  • From vorlon@21:1/195.1 to paulie420 on Saturday, July 09, 2022 11:47:21
    Hi paulie420,

    Ahhh, thanks for reminding my about the fact that all PVE tools have
    CLI commands to do all the things most ppl use the web interface for -
    I was going to use a RPi backup server machine to do the backups, but
    yea - I could just tell PVE to send them over... awesome.

    Maybe using PVE .vma.gz backups will be enough for me - I don't know
    that I *need* an entire PBS system.

    When I started with proxmox, the PBS system didn't exist, so had to get
    down and dirty with some scripting.

    Unless you have more complex requirements, a vma.gz backup is rather
    powerfull. No extra work, and when I've done restore testing, it's just
    worked.

    Ok - now this is just overkill!!! Thanks a ton for all the data here,
    I'll be sure to extract this message and use it to help setup my
    solution. GREAT STUFF - thanks, vorlon!

    NP, what I posted was a cut down version of what I'ge got running on my
    proxmox server (Btw: It's 1000Km away, and so is the remote part of this).

    I figured you didn't need info for seting up the nfs part from previous messages.



    \/orlon
    aka
    Stephen


    --- Talisman v0.42-dev (Linux/m68k)
    * Origin: Vorlon Empire: Amiga 3000 powered in Sector 550 (21:1/195.1)
  • From 2twisty@21:3/166 to vorlon on Monday, July 11, 2022 09:06:26
    That's one thing that would make a pi so much better, is if they had two sata ports.

    If you are using the Compute Module, you can get backplanes that have SATA or even a 1x PCIe slot that you could put a sata HBA in. Jeff Geerling has done that, but even then, I think ZFS is WAAAAAY too big a system for a Pi's compute power.

    --- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
    * Origin: The Ratrace Losers (21:3/166)
  • From vorlon@21:1/195.1 to 2twisty on Tuesday, July 12, 2022 10:36:24
    Hi 2twisty,

    That's one thing that would make a pi so much better, is if they
    had two sata ports.

    If you are using the Compute Module, you can get backplanes that have
    SATA or even a 1x PCIe slot that you could put a sata HBA in. Jeff
    Geerling has done that, but even then, I think ZFS is WAAAAAY too big
    a system for a Pi's compute power.

    That wont stop someone from trying though.! For example I have setup
    Talisman on linux for M68K even though apam didn't take that into account
    when coding the bbs. He was open to making things work though.



    \/orlon
    aka
    Stephen


    --- Talisman v0.42-dev (Linux/m68k)
    * Origin: Vorlon Empire: Amiga 3000 powered in Sector 550 (21:1/195.1)