Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

embroider5169
Posts: 57
Joined: Wed Jul 19, 2023 5:46 pm

Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by embroider5169 » Tue Jun 04, 2024 8:20 am

Before we start i expect you to have the basic knowledge of rclone and what it is.

1) First we need to create a rclone config file, the below example can be copied as a template:

Code: Select all

[blomp]
type = swift
user = changeme@email.com
key = changemePassword
auth = https://authenticate.ain.net
tenant = storage
auth_version = 2
endpoint_type = public
leave_parts_on_error = true
chunk_size = 1P
no_chunk = false

[blomp_encrypted]
type = crypt
remote = blomp:changeme@email.com
password = changemePassword

[blomp_chunker]
type = chunker
remote = blomp_encrypted:
hash_type = none
name_format = *.#####
transactions = norename
chunk_size = 4800Mi
In the first part of the template you have to change the following settings:
[blomp]
user = changeme@email.com --> change this to your e-mail used with blomp
key = changemePassword --> change this to your password used with blomp
The second part we have to change is the following:
[blomp_encrypted]
remote = blomp:changeme@email.com --> Change this again but leave the “blomp:” part.
password = changemePassword → Change this to a encrypted password (dont use special characters)
The third part is a chunker, chunker is only needed if you upload files bigger than 5GB as the SWIFT protocol does not allow file size above 5GB.
[blomp_chunker]
name_format = *.##### --> The # (hashtag) is the amount of numers behind a file (example filename.zip.00001 filename.zip.00002 etc.)
chunk_size = 4800Mi --> this can be changed to any file size below 5GB, for safety use 4800Mi (MB)
Now save the changes to a file with the extension .conf
For this guide i will simply use rcloneblomp.conf

I’m a linux user, so some flags might not work on windows / osx, please refer to the rclone documentation to see what works and what does not work depending on your OS --> https://rclone.org/flags/

Now for simplicity i have my folder and configuration file in the same directy to keep it simple
config: rcloneblomp.conf
foldername: blompmount

mount command:

Code: Select all

rclone mount blomp_chunker: blompmount/ --config rcloneblomp.conf --allow-other --vfs-cache-mode=writes --poll-interval 0 --dir-cache-time 1h --use-server-modtime --no-checksum --vfs-fast-fingerprint --vfs-cache-max-size=5G --vfs-read-chunk-size=16M --vfs-read-chunk-size-limit=1G --buffer-size=16M --transfers 1 --vfs-disk-space-total-size 10T -v
Explaining all the flags:
The rcloune mount is called:
blomp_chunker:
This refers to where the config file is we just created.
--config xxx
Allowing other users / applications to use the mount.
--allow-other
I picked writes, as off, minimal and full kept crashing the mount, also Swift does not support off, minimal at all.
--vfs-cache-mode writes
Polling is not supported on SWIFT protocol --> 0 = disabled.
--poll-interval 0
How long folder structure should be cached before refreshing, you can set it up to 8700h (1year ish.)
--dir-cache-time 1h
Needed as SWIFT protocol does not support support reading local modtime
--use-server-modtime
Needed as SWIFT protocol does not support checksum data.
--no-checksum
Just show/read the file, should be quicker on SWIFT protocol according to rclone (optional)
--vfs-fast-fingerprint
The amount of data cached on your hard drive for quick access. If files are bigger than 5GB it will use more than 5GB, after closing a file unused cache will be deleted.
--vfs-cache-max-size 5G
Read any file in parts of 16MB increasing each read (16/32/64 etc)
--vfs-read-chunk-size=16M
Max read chunk size is 1GB, so starting at 16/32/64 etc all the way up to 1024 and then keeps grabbing 1GB)
--vfs-read-chunk-size-limit=1G
Try putting read-chunk-size into RAM memory instead of HDD memory.
--buffer-size=16MB
This affects uploads, not downloads, SWIFT only supports 1 for upload. (if you change this and you use the -vv flag you will get many errors, it will keep retrying but will still only upload 1 file and then the next).
--transfers 1
This changed the 1P space to the 10TB, change this number to your subscription, this is just a visual update (optional)
--vfs-disk-space-total-size 10T
Last edited by embroider5169 on Tue Jun 04, 2024 8:27 am, edited 1 time in total.



BlompSupport
Posts: 711
Joined: Wed Feb 13, 2019 4:20 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by BlompSupport » Wed Jun 12, 2024 8:54 am

Dear embroider5169,

We want to take this opportunity to thank you for sharing your guide on setting up Rclone, including detailed instructions on creating the Rclone config and mounting Rclone with Blomp. Your contribution is greatly appreciated by the community, and we are thrilled to have you as a part of our Blomp family! <3

Wishing you a great day ahead!

Best Regards,
- Blomp support

Check out our video! Trust me, you're going to blomping love it. https://www.youtube.com/watch?v=klnAFs3H3bw&t=70s

embroider5169
Posts: 57
Joined: Wed Jul 19, 2023 5:46 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by embroider5169 » Thu Jun 13, 2024 7:35 am

BlompSupport wrote:
Wed Jun 12, 2024 8:54 am
Dear embroider5169,

We want to take this opportunity to thank you for sharing your guide on setting up Rclone, including detailed instructions on creating the Rclone config and mounting Rclone with Blomp. Your contribution is greatly appreciated by the community, and we are thrilled to have you as a part of our Blomp family! <3

Wishing you a great day ahead!

Best Regards,
Thank you for the kind words! i hope you can assist me with a small thing, change the title

From:
Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)
To:
Community guide – Basic rclone setup.

i will maintain this post with new chances or feedback received from community and update the post accordingly.

For some reason im unable to change the title of the post, so hoping you can help with that.

Cheers.

gian1
Posts: 1
Joined: Fri Nov 01, 2024 9:55 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by gian1 » Sun Nov 03, 2024 10:57 am

Hi, embroider5169

Thanks for the guide in rclone setup. However, can I clarify this with you?

Im also using rclone in Linux. I'm trying to follow your instructions but I keep getting these errors:

ERROR : Swift root: Statfs failed: container listing failed: Operation forbidden
ERROR : IO error: container listing failed: Operation forbidden

Might is be related to the password(s) I'm using? I;m not sure what I should use in this part:

[blomp_encrypted]
password = changemePassword → Change this to a encrypted password (dont use special characters)

Should this be the same as the account password that I signed up for my account with? Otherwise, how do I determine wat encrypted password it should be?

Thanks a lot

mvjunkie
Posts: 6
Joined: Tue Apr 16, 2024 3:40 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by mvjunkie » Sun Nov 03, 2024 9:39 pm

gian1 wrote:
Sun Nov 03, 2024 10:57 am
Hi, embroider5169

Thanks for the guide in rclone setup. However, can I clarify this with you?

Im also using rclone in Linux. I'm trying to follow your instructions but I keep getting these errors:

ERROR : Swift root: Statfs failed: container listing failed: Operation forbidden
ERROR : IO error: container listing failed: Operation forbidden

Might is be related to the password(s) I'm using? I;m not sure what I should use in this part:

[blomp_encrypted]
password = changemePassword → Change this to a encrypted password (dont use special characters)

Should this be the same as the account password that I signed up for my account with? Otherwise, how do I determine wat encrypted password it should be?

Thanks a lot
I am getting the same error, its the auth URL. In the guide it is; https://authenticate.ain.net

However, from what little I can gather, this has been changed to a new URL, I tried to look for it last night for an hour and coud not find it. As soon as you replace https://authenticate.ain.net with the updated URL, you should be good to go.

BlompSupport
Posts: 711
Joined: Wed Feb 13, 2019 4:20 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by BlompSupport » Mon Nov 04, 2024 12:32 pm

Hey mvjunkie,

We're really sorry to hear you ran into some error messages while setting up Rclone! :?
We appreciate your patience as we look into this, and we’ll keep you posted as soon as we have a solution.

Thanks so much for choosing Blomp!

Best Regards,
- Blomp support

Check out our video! Trust me, you're going to blomping love it. https://www.youtube.com/watch?v=klnAFs3H3bw&t=70s

BlompSupport
Posts: 711
Joined: Wed Feb 13, 2019 4:20 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by BlompSupport » Thu Nov 07, 2024 10:11 am

gian1 wrote:
Sun Nov 03, 2024 10:57 am
Hi, embroider5169

Thanks for the guide in rclone setup. However, can I clarify this with you?

Im also using rclone in Linux. I'm trying to follow your instructions but I keep getting these errors:

ERROR : Swift root: Statfs failed: container listing failed: Operation forbidden
ERROR : IO error: container listing failed: Operation forbidden

Might is be related to the password(s) I'm using? I;m not sure what I should use in this part:

[blomp_encrypted]
password = changemePassword → Change this to a encrypted password (dont use special characters)

Should this be the same as the account password that I signed up for my account with? Otherwise, how do I determine wat encrypted password it should be?

Thanks a lot
Hello embroider5169,
How are you? :) We have a green light from the tech team that the issue is fixed.
Can you please check and confirm it? Please make sure that you use the following auth URL https://authenticate.ain.net.

Thank you.

Regards,
- Blomp support

Check out our video! Trust me, you're going to blomping love it. https://www.youtube.com/watch?v=klnAFs3H3bw&t=70s

mvjunkie
Posts: 6
Joined: Tue Apr 16, 2024 3:40 pm

Re: Community guide – Basic rclone setup (will update in the future with rclone docker volume plugin)

Post by mvjunkie » Thu Nov 07, 2024 5:36 pm

BlompSupport wrote:
Thu Nov 07, 2024 10:11 am
gian1 wrote:
Sun Nov 03, 2024 10:57 am
Hi, embroider5169

Thanks for the guide in rclone setup. However, can I clarify this with you?

Im also using rclone in Linux. I'm trying to follow your instructions but I keep getting these errors:

ERROR : Swift root: Statfs failed: container listing failed: Operation forbidden
ERROR : IO error: container listing failed: Operation forbidden

Might is be related to the password(s) I'm using? I;m not sure what I should use in this part:

[blomp_encrypted]
password = changemePassword → Change this to a encrypted password (dont use special characters)

Should this be the same as the account password that I signed up for my account with? Otherwise, how do I determine wat encrypted password it should be?

Thanks a lot
Hello embroider5169,
How are you? :) We have a green light from the tech team that the issue is fixed.
Can you please check and confirm it? Please make sure that you use the following auth URL https://authenticate.ain.net.

Thank you.

Regards,
This URL does not work in rclone, when you attempt to use it, and then try to connect your get an error;
" 2024/11/07 16:33:25 ERROR : Swift root: Statfs failed: container listing failed: Operation forbidden "

However after finding someone on reddit to help me, they told me to use " https://authenticate.blomp.com/v2.0/ ", and rclone is finally working.

Dear Blomp, is this the correct ( new ) url to be using, or am I logging into " https://authenticate.ain.net " incorrectly ?

Thanks

Post Reply