Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post Reply
embroider5169
Posts: 57
Joined: Wed Jul 19, 2023 5:46 pm

Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by embroider5169 » Tue Jun 04, 2024 8:20 am

Before we start i expect you to have the basic knowledge of rclone and what it is.

1) First we need to create a rclone config file, the below example can be copied as a template:

Code: Select all

[blomp]
type = swift
user = changeme@email.com
key = changemePassword
auth = https://authenticate.ain.net
tenant = storage
auth_version = 2
endpoint_type = public
leave_parts_on_error = true
chunk_size = 1P
no_chunk = false

[blomp_encrypted]
type = crypt
remote = blomp:changeme@email.com
password = changemePassword

[blomp_chunker]
type = chunker
remote = blomp_encrypted:
hash_type = none
name_format = *.#####
transactions = norename
chunk_size = 4800Mi
In the first part of the template you have to change the following settings:
[blomp]
user = changeme@email.com --> change this to your e-mail used with blomp
key = changemePassword --> change this to your password used with blomp
The second part we have to change is the following:
[blomp_encrypted]
remote = blomp:changeme@email.com --> Change this again but leave the “blomp:” part.
password = changemePassword → Change this to a encrypted password (dont use special characters)
The third part is a chunker, chunker is only needed if you upload files bigger than 5GB as the SWIFT protocol does not allow file size above 5GB.
[blomp_chunker]
name_format = *.##### --> The # (hashtag) is the amount of numers behind a file (example filename.zip.00001 filename.zip.00002 etc.)
chunk_size = 4800Mi --> this can be changed to any file size below 5GB, for safety use 4800Mi (MB)
Now save the changes to a file with the extension .conf
For this guide i will simply use rcloneblomp.conf

I’m a linux user, so some flags might not work on windows / osx, please refer to the rclone documentation to see what works and what does not work depending on your OS --> https://rclone.org/flags/

Now for simplicity i have my folder and configuration file in the same directy to keep it simple
config: rcloneblomp.conf
foldername: blompmount

mount command:

Code: Select all

rclone mount blomp_chunker: blompmount/ --config rcloneblomp.conf --allow-other --vfs-cache-mode=writes --poll-interval 0 --dir-cache-time 1h --use-server-modtime --no-checksum --vfs-fast-fingerprint --vfs-cache-max-size=5G --vfs-read-chunk-size=16M --vfs-read-chunk-size-limit=1G --buffer-size=16M --transfers 1 --vfs-disk-space-total-size 10T -v
Explaining all the flags:
The rcloune mount is called:
blomp_chunker:
This refers to where the config file is we just created.
--config xxx
Allowing other users / applications to use the mount.
--allow-other
I picked writes, as off, minimal and full kept crashing the mount, also Swift does not support off, minimal at all.
--vfs-cache-mode writes
Polling is not supported on SWIFT protocol --> 0 = disabled.
--poll-interval 0
How long folder structure should be cached before refreshing, you can set it up to 8700h (1year ish.)
--dir-cache-time 1h
Needed as SWIFT protocol does not support support reading local modtime
--use-server-modtime
Needed as SWIFT protocol does not support checksum data.
--no-checksum
Just show/read the file, should be quicker on SWIFT protocol according to rclone (optional)
--vfs-fast-fingerprint
The amount of data cached on your hard drive for quick access. If files are bigger than 5GB it will use more than 5GB, after closing a file unused cache will be deleted.
--vfs-cache-max-size 5G
Read any file in parts of 16MB increasing each read (16/32/64 etc)
--vfs-read-chunk-size=16M
Max read chunk size is 1GB, so starting at 16/32/64 etc all the way up to 1024 and then keeps grabbing 1GB)
--vfs-read-chunk-size-limit=1G
Try putting read-chunk-size into RAM memory instead of HDD memory.
--buffer-size=16MB
This affects uploads, not downloads, SWIFT only supports 1 for upload. (if you change this and you use the -vv flag you will get many errors, it will keep retrying but will still only upload 1 file and then the next).
--transfers 1
This changed the 1P space to the 10TB, change this number to your subscription, this is just a visual update (optional)
--vfs-disk-space-total-size 10T
Last edited by embroider5169 on Tue Jun 04, 2024 8:27 am, edited 1 time in total.



BlompSupport
Posts: 825
Joined: Wed Feb 13, 2019 4:20 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by BlompSupport » Wed Jun 12, 2024 8:54 am

Dear embroider5169,

We want to take this opportunity to thank you for sharing your guide on setting up Rclone, including detailed instructions on creating the Rclone config and mounting Rclone with Blomp. Your contribution is greatly appreciated by the community, and we are thrilled to have you as a part of our Blomp family! <3

Wishing you a great day ahead!

Best Regards,
- Blomp support

Check out our video! Trust me, you're going to blomping love it. https://www.youtube.com/watch?v=klnAFs3H3bw&t=70s

embroider5169
Posts: 57
Joined: Wed Jul 19, 2023 5:46 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by embroider5169 » Thu Jun 13, 2024 7:35 am

BlompSupport wrote:
Wed Jun 12, 2024 8:54 am
Dear embroider5169,

We want to take this opportunity to thank you for sharing your guide on setting up Rclone, including detailed instructions on creating the Rclone config and mounting Rclone with Blomp. Your contribution is greatly appreciated by the community, and we are thrilled to have you as a part of our Blomp family! <3

Wishing you a great day ahead!

Best Regards,
Thank you for the kind words! i hope you can assist me with a small thing, change the title

From:
Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)
To:
Community guide – Basic rclone setup.

i will maintain this post with new chances or feedback received from community and update the post accordingly.

For some reason im unable to change the title of the post, so hoping you can help with that.

Cheers.

Post Reply