Skip to content

Slow loading files fail (over 1 hour) #77

@GoogleCodeExporter

Description

@GoogleCodeExporter
What steps will reproduce the problem?
1. Load a large file, or load over a slow network
2. Watch the the tempfile table in the database
3. Watch the actual file on the storage node

What is the expected output? What do you see instead?
I expect my slow loading file to be loaded.

After an hour the tempfile entry disappears and the file on the storage node is 
deleted. My load thread completes but MogileFS never responds to create_close 
command (timed out after 10 mins for me) and the file is not present.

I'm no good with perl but I think the problem seems to be in the Delete worker:
    # dig up some temporary files to purge
    my $sto = Mgd::get_store();
    my $too_old = int($ENV{T_TEMPFILE_TOO_OLD} || 3600);
    my $tempfiles = $sto->old_tempfiles($too_old);
    return 0 unless $tempfiles && @$tempfiles;

Fixes
 - it would be best if the worker could check to see if the file is still been written to before culling it so I don't need to raise the value.
 - If I have to raise the value can we set it in the config file rather than an environmental variable?

Thanks!

Original issue reported on code.google.com by simon...@gmail.com on 20 Mar 2013 at 9:33

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions