Rcloud Error: NOTICE: Failed to move: Too Large Object

[logvin]

Running an Ubuntu box. Been using rClone for years with various cloud storage.

I can connected to my blomp-crypt, which connects to the chunker and then to the remote. I can create, modify, and delete files, which are encrypted and I can see the file on the blomp website, with an encrypted name of course. That's all working great. I tried to move a file to blomp-crypt endpoint and if fails with the Too Large Object.

I imagine it has to do with the chunker stuff, but I'm not very familiar with it. Again, this setup is working for reading and uploading smaller files, but when they get larger they just get stuck in vfs cache.

Code: Select all

[blomp-remote]
type = swift
user = my.email@domain.com
key = redacted
auth = https://authenticate.ain.net
tenant = storage
auth_version = 2
leave_parts_on_error = true
chunk_size = 1P
no_chunk = false

[blomp-chunker]
type = alias
remote = blomp-remote:my.email@domain.com/
chunk_size = 5242879K
hash_type = none
name_format = *.#####
start_from = 1
meta_format = simplejson
fail_hard = false
transactions = norename

[blomp-crypt]
type = crypt
remote = blomp-chunker:
filename_encryption = standard
password = redacted1
password2 = redacted2

Here is my rclone-blomp.service:

Code: Select all

 [Unit]
 Description="Blomp-Crypt"
 After=network-online.target
 Wants=network-online.target
 
 [Service]
 User=plex
 Group=plexapps
 TimeoutStartSec=6000
 
 Type=notify
 ExecStart=/usr/bin/rclone mount blomp-crypt: /local/storage/mounts/blomp \
                 --config /local/storage/rclone.conf \
                 --no-modtime \
                 --no-checksum \
                 --no-seek \
                 --allow-non-empty \
                 --allow-other \
                 --vfs-cache-mode full \
                 --vfs-cache-max-age 10m \
                 --bwlimit-file 16M \
                 --poll-interval 30s \
                 --dir-cache-time 60m \
                 --transfers 8 \
                 --checkers 8 \
                 --file-perms 660 \
                 --dir-perms 770 \
                 --uid 1004 \
                 --gid 1003 \
                 --log-level DEBUG \
                 --log-file /local/storage/logs/blomp_logs.log \
                 --cache-dir /local/storage/cache/blomp \
                 --log-systemd \
                 --write-back-cache \
                 --use-mmap \
                 --stats 1m \
                 --timeout 1m \
                 --contimeout 10s \
                 --fast-list
 ExecStop=/bin/fusermount -uz /local/storage/mounts/blomp
 Restart=on-abort
 RestartSec=10
 
 [Install]
 WantedBy=multi-user.target

Logs from rclone-blomp.service:

Code: Select all

DEBUG : Encrypted drive 'blomp-crypt:': vfs cache RemoveNotInUse (maxAge=600000000000, emptyOnly=false): item Videos/Movies/Home_Video_1.mkv not removed, freed 0 bytes
INFO  : Encrypted drive 'blomp-crypt:': vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 10.038Gi (was 10.038Gi)                                                           
INFO  : 
    Transferred:              21.625 MiB / 21.625 MiB, 100%, 0 B/s, ETA -
    Errors:                 7 (retrying may help)
    Checks:                 0 / 0, -, Listed 14
    Elapsed time:     13m57.9s

[BlompSupport]

Hi logvin,

Thank you so much for reaching out and for giving Blomp a try. We’re glad to hear you’re exploring it with rclone as many of our users do the same, and we’re here to help.

Our team of expert developers is now looking into this issue. In the meantime, if you can share any extra details like error messages or logs, it will help us speed things up.

We’re really happy to have you here with us!

Warm regards,

[BlompSupport]

Hi logvin,

From what we can see, the issue seems to be with the cache timing. The logs indicate that the file transfer took over 13 minutes, but the cache’s maximum age is currently set to only 10 minutes. This mismatch likely caused the transfer to fail.

We suggest increasing the --vfs-cache-max-age value to a larger time so the cache doesn’t expire before the file completes transferring. That adjustment should prevent the file from getting stuck.

Please give this a try as it should resolve the issue. We’re here to help if you run into anything else!

Warm regards,