format for config times is inconsistent

[joooeey]

My luigi.cfg looked like this:

[core]
# Hide DeprecationWarning:
autoload_range: false

[scheduler]
# Time before a finished task graph is deleted (in seconds):
remove_delay: 10

[worker]
# Keep workers alive as long as they have pending jobs
keep_alive: true
max_keep_alive_idle_duration: 3600

Now luigi can parse the 10 seconds but it can't pass the 3600 seconds. That's surprising.

Turns out I had to specify max_keep_alive_idle_duration: 1 hour but that can't be found in the documentation without inspecting the source code. Yes, it's in the docs for the TimedeltaParameter but nowhere does it say that this option is interpreted as a TimedeltaParameter.

However, it would be better if all the config values specifying a time can be given in the same format. I'll see if I can get in a PR to solve this. The idea would be for all times to accept all formats accepted by TimeDeltaParameter plus to interpret an integer as seconds.

[lallea]

Thanks for reporting inconsistencies. If you make a PR for this, could you consider adding the time unit to the config parameter names, e.g. remove_delay_seconds? IMHO that decreases risk for config mistakes. I realise that it makes the PR scope larger, since it would be necessary to keep the old names for compatibility. But worth consideration.

[joooeey]

I don't like that idea. I'm making it so that you can also enter something like remove_delay: 1 hour 30 minutes or the seconds as float. I think I'll keep the names as they are but perhaps it would make sense to actually remove the unit then.

[joooeey]

I was looking into it and seems what I was proposing is way too much effort because it would break a bunch of tests. The idea was to create a hybrid between TimeDeltaParameter and FloatParameter.

New proposal:

• change TimedeltaParameters like max_keep_alive_idle_duration to also accept a float or int input and interpret it as seconds.
• mention the word "seconds" for every parameter in the docs where it's applicable
• but keep the names of the parameters as they are