Bug #3019

Failed to generate resources / Could not retieve file metadata

Added by Markus Roberts about 2 years ago. Updated 4 months ago.

Status:Needs More Information Start date:01/08/2010
Priority:Normal Due date:
Assignee:- % Done:

0%

Category:fileserving
Target version:-
Affected Puppet version:0.24.4 Branch:
Keywords:
Votes: 4

Description

(Reported by Mark Plaksin)

I think I’m getting 3 different stack traces on our RHEL 4.7 box. (1) has “connect” at the top and is about a recursive file resource. (2) and (3) both start with “fail”. In (2) the file on the file server is symlink (and is inside a recursive file resource, in (3) the file is a normal file. The traces don’t always happen on the same files from run to run.

------------------------------------------------------------------------------
1:

/usr/lib/ruby/1.8/net/protocols.rb:49:in `connect'
/usr/lib/ruby/1.8/net/protocols.rb:49:in `ssl_connect'
/usr/lib/ruby/1.8/net/https.rb:181:in `on_connect'
/usr/lib/ruby/1.8/net/http.rb:433:in `do_start'
/usr/lib/ruby/1.8/net/http.rb:419:in `start'
/usr/lib/ruby/1.8/net/http.rb:824:in `request'
/usr/lib/ruby/1.8/net/http.rb:618:in `get'
/usr/src/git/puppet/lib/puppet/indirector/rest.rb:73:in `search'
/usr/src/git/puppet/lib/puppet/indirector/indirection.rb:240:in `search'
/usr/src/git/puppet/lib/puppet/indirector.rb:59:in `search'
/usr/src/git/puppet/lib/puppet/type/file.rb:595:in `perform_recursion'
/usr/src/git/puppet/lib/puppet/type/file.rb:562:in `recurse_remote'
/usr/src/git/puppet/lib/puppet/type/file.rb:561:in `collect'
/usr/src/git/puppet/lib/puppet/type/file.rb:561:in `recurse_remote'
/usr/src/git/puppet/lib/puppet/type/file.rb:483:in `recurse'
/usr/src/git/puppet/lib/puppet/type/file.rb:385:in `eval_generate'
/usr/src/git/puppet/lib/puppet/transaction.rb:349:in `send'
/usr/src/git/puppet/lib/puppet/transaction.rb:349:in `generate_additional_resources'
/usr/src/git/puppet/lib/puppet/transaction.rb:193:in `eval_generate'
/usr/src/git/puppet/lib/puppet/transaction.rb:240:in `eval_children_and_apply_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:207:in `eval_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:296:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:295:in `thinmark'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `thinmark'
/usr/src/git/puppet/lib/puppet/transaction.rb:297:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:289:in `collect'
/usr/src/git/puppet/lib/puppet/transaction.rb:289:in `evaluate'
/usr/src/git/puppet/lib/puppet/resource/catalog.rb:142:in `apply'
/usr/src/git/puppet/lib/puppet/configurer.rb:153:in `run'
/usr/src/git/puppet/lib/puppet/configurer.rb:152:in `benchmark'
/usr/src/git/puppet/lib/puppet/util.rb:177:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:177:in `benchmark'
/usr/src/git/puppet/lib/puppet/configurer.rb:152:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `lock'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `synchronize'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:51:in `with_client'
/usr/src/git/puppet/lib/puppet/agent.rb:58:in `run'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:103:in `onetime'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:94:in `onetime'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `send'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `run_command'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `exit_on_fail'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
sbin/puppetd:159
err: //allusers::bjones/Allusers::Userfile[bjones/bin]/File[/home/bjones/bin]: Failed to generate additional resources using 'eval_generate':

NOTE: There’s nothing after that final colon except for a single space.

------------------------------------------------------------------------------
2:

/usr/src/git/puppet/lib/puppet/parameter.rb:401:in `fail'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:157:in `init_metadata'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:149:in `each'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:149:in `init_metadata'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:146:in `init_metadata'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:106:in `send'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:106:in `cached_value'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:46:in `metadata'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:45:in `metadata'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:111:in `copy_source_values'
/usr/src/git/puppet/lib/puppet/type/file.rb:630:in `retrieve'
/usr/src/git/puppet/lib/puppet/type.rb:726:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:62:in `apply'
/usr/src/git/puppet/lib/puppet/transaction.rb:251:in `eval_children_and_apply_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:250:in `thinmark'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `thinmark'
/usr/src/git/puppet/lib/puppet/transaction.rb:252:in `eval_children_and_apply_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:207:in `eval_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:296:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:295:in `thinmark'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `thinmark'
/usr/src/git/puppet/lib/puppet/transaction.rb:297:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:289:in `collect'
/usr/src/git/puppet/lib/puppet/transaction.rb:289:in `evaluate'
/usr/src/git/puppet/lib/puppet/resource/catalog.rb:142:in `apply'
/usr/src/git/puppet/lib/puppet/configurer.rb:153:in `run'
/usr/src/git/puppet/lib/puppet/configurer.rb:152:in `benchmark'
/usr/src/git/puppet/lib/puppet/util.rb:177:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:177:in `benchmark'
/usr/src/git/puppet/lib/puppet/configurer.rb:152:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `lock'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `synchronize'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:51:in `with_client'
/usr/src/git/puppet/lib/puppet/agent.rb:58:in `run'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:103:in `onetime'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:94:in `onetime'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `send'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `run_command'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `exit_on_fail'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
sbin/puppetd:159
err: //allusers::tss/Allusers::Userfile[tss/.screenrc]/File[/home/tss/.screenrc]: Failed to retrieve current state of resource: Could not retrieve file metadata for puppet://billy.bor.usg.edu/allusers/tss/.screenrc: at /usr/src/git/bor/puppet/modules/allusers/manifests/init.pp:23
------------------------------------------------------------------------------
3:
/usr/src/git/puppet/lib/puppet/parameter.rb:401:in `fail'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:157:in `init_metadata'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:149:in `each'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:149:in `init_metadata'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:146:in `init_metadata'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:106:in `send'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:106:in `cached_value'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:46:in `metadata'
/usr/src/git/puppet/lib/puppet/util/cacher.rb:45:in `metadata'
/usr/src/git/puppet/lib/puppet/type/file/source.rb:111:in `copy_source_values'
/usr/src/git/puppet/lib/puppet/type/file.rb:630:in `retrieve'
/usr/src/git/puppet/lib/puppet/type.rb:726:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:62:in `apply'
/usr/src/git/puppet/lib/puppet/transaction.rb:251:in `eval_children_and_apply_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:250:in `thinmark'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `thinmark'
/usr/src/git/puppet/lib/puppet/transaction.rb:252:in `eval_children_and_apply_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:207:in `eval_resource'
/usr/src/git/puppet/lib/puppet/transaction.rb:296:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:295:in `thinmark'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `thinmark'
/usr/src/git/puppet/lib/puppet/transaction.rb:297:in `evaluate'
/usr/src/git/puppet/lib/puppet/transaction.rb:289:in `collect'
/usr/src/git/puppet/lib/puppet/transaction.rb:289:in `evaluate'
/usr/src/git/puppet/lib/puppet/resource/catalog.rb:142:in `apply'
/usr/src/git/puppet/lib/puppet/configurer.rb:153:in `run'
/usr/src/git/puppet/lib/puppet/configurer.rb:152:in `benchmark'
/usr/src/git/puppet/lib/puppet/util.rb:177:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:177:in `benchmark'
/usr/src/git/puppet/lib/puppet/configurer.rb:152:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `lock'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `synchronize'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:51:in `with_client'
/usr/src/git/puppet/lib/puppet/agent.rb:58:in `run'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:103:in `onetime'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:94:in `onetime'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `send'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `run_command'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `exit_on_fail'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
sbin/puppetd:159
err: //networker/File[/usr/local/bin/NSRpre_process]: Failed to retrieve current state of resource: Could not retrieve file metadata for puppet://billy.bor.usg.edu/networker/NSRpre_process: at /usr/src/git/bor/puppet/modules/networker/manifests/init.pp:26

Related issues

related to Puppet - Bug #3083: Failed to retrieve current state of resource: undefined m... Closed 01/17/2010

History

Updated by Markus Roberts about 2 years ago

Mark —

I tried to reproduce this on billy using the go.server / go.client scripts and did not see it or anything like it. Could you provide any hints as to how to reproduce this?

— Markus

Updated by Mark Plaksin about 2 years ago

billy’s running RHEL5 and is the server, squier is RHEL4. I got these running go.client on squier with your branch for #3013.

Updated by Markus Roberts about 2 years ago

Thanks, I’ll try to replicate.

Updated by Markus Roberts about 2 years ago

No luck so far, but I did get:

/usr/lib/ruby/1.8/net/protocols.rb:49:in `connect'
/usr/lib/ruby/1.8/net/protocols.rb:49:in `ssl_connect'
/usr/lib/ruby/1.8/net/https.rb:181:in `on_connect'
/usr/lib/ruby/1.8/net/http.rb:433:in `do_start'
/usr/lib/ruby/1.8/net/http.rb:419:in `start'
/usr/lib/ruby/1.8/net/http.rb:824:in `request'
/usr/lib/ruby/1.8/net/http.rb:618:in `get'
/usr/src/git/puppet/lib/puppet/indirector/rest.rb:69:in `find'
/usr/src/git/puppet/lib/puppet/indirector/indirection.rb:195:in `find'
/usr/src/git/puppet/lib/puppet/indirector.rb:51:in `find'
/usr/src/git/puppet/lib/puppet/configurer.rb:94:in `retrieve_catalog'
/usr/src/git/puppet/lib/puppet/configurer.rb:93:in `thinmark'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `measure'
/usr/lib/ruby/1.8/benchmark.rb:342:in `realtime'
/usr/src/git/puppet/lib/puppet/util.rb:405:in `thinmark'
/usr/src/git/puppet/lib/puppet/configurer.rb:95:in `retrieve_catalog'
/usr/src/git/puppet/lib/puppet/configurer.rb:146:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `lock'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `synchronize'
/usr/src/git/puppet/lib/puppet/agent.rb:53:in `run'
/usr/src/git/puppet/lib/puppet/agent.rb:51:in `with_client'
/usr/src/git/puppet/lib/puppet/agent.rb:58:in `run'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:103:in `onetime'
/usr/src/git/puppet/lib/puppet/application/puppetd.rb:94:in `onetime'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `send'
/usr/src/git/puppet/lib/puppet/application.rb:226:in `run_command'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `exit_on_fail'
/usr/src/git/puppet/lib/puppet/application.rb:217:in `run'
sbin/puppetd:159
err: Could not retrieve catalog from remote server: block type is not 01
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

on one attempt. I’ll be trying again this morning.

Updated by Markus Roberts about 2 years ago

  • Status changed from Investigating to Needs More Information
  • Assignee changed from Markus Roberts to Mark Plaksin

I get the above error even with 0.25.1; have you seen this previosly or do you have any idea what I may be doing wrong?

Updated by Markus Roberts about 2 years ago

  • Target version changed from 0.25.3 to 0.25.4

Updated by Mark Plaksin about 2 years ago

I think that 0.25.1 on RHEL4 had other problems which may have masked the ones that cropped up when I tested your #3013 branch. I just retested with 0.25.3 on both the client (RHEL4) and the server (RHEL5) and got the same “block type” error as you saw.

As I mentioned via email, we’re suddenly and unexpectedly not running any more RHEL4 boxes (except for a few stragglers which will got to RHEL5 soon). But I’m happy to help debug further.

Updated by James Turnbull about 2 years ago

  • Target version changed from 0.25.4 to 0.25.5

Updated by Markus Roberts about 2 years ago

Mark —

I’ve been working the block type errors while Jesse tries to glean something from the stack traces. It appears that the block errors are a problem with the certificates, and when I retry starting with a clean ssl directory (see go.server.mqr and go.client.mqr) I do not get them anymore.

However it now fails on the puppetmaster (squier) with:

err: Failed to find billy.bor.usg.edu via exec: Execution of '/usr/local/bin/node billy.bor.usg.edu' returned 1: No such file or directory - /usr/local/bin/node

which doesn’t seem related to the initial error either.

Thoughts?

— Markus

Updated by Jesse Wolfe about 2 years ago

It looks to me like the errors either occurred on the server (and perhaps there’s useful debug information there) or on the network (which would suggest that our error messages aren’t conveying enough information)

Updated by Mark Plaksin about 2 years ago

Hi:

billy (RHEL5) is the server. squier (RHEL4) is the client. We’ve never run the server on RHEL4 or squier. The “/usr/local/bin/node not found” message must have come from running the server on squier.

I just created go.client.49 and pointed it at a different SSL directory and signed squier’s new cert on billy. When I run that against the server billy on billy I get sporadic stack traces that seem to match the ones in this ticket. I also get this message:

notice: //puppet/Cron[sleepyRestartPuppet]/ensure: created “–”:6: bad command errors in crontab file, can’t install.

And the cron job isn’t created.

I’m not sure what happened with the old SSL dir. We had used been using it to test against billy for a while and it worked fine. Then those “01” errors started showing up.

Updated by Mark Plaksin about 2 years ago

  • Assignee changed from Mark Plaksin to Markus Roberts
  • Affected Puppet version changed from 0.25.2 to 0.25.4rc2

With 0.24.5rc2 I get the same behavior as described in #11.

Updated by Josh Endries about 2 years ago

I get these errors (mostly eval_generate) all the time on my hosts (WEBRick). It’s sporadic, doesn’t happen every run. It seems like it happens in modules that deal with many small files (Tomcat is ~2100 and Fedora is ~1800). I have been dealing with it (ok ok, ignoring it) for months and haven’t found any pattern to the errors yet. Some have error messages, some don’t. Most are usually EOF errors, which would make sense to me if there were a problem in a manifest but most of the time and whenever I run it manually the same host seems to work fine. I’m 99% certain networking is fine as it’s only one or two hosts that complain when I have dozens, and if the network were flaky I’d have all sorts of other errors. This is on 0.25.1. I haven’t ran the server in debug mode yet but I’ll set that up today and hope one of these comes through quickly. If this is ticket hijacking let me know and ignore me. :)

Mon Jan 25 07:35:07 -0500 2010 //Node[repo1]/Fedora[fedora]/Tomcat[fedora]/File[/usr/local/tomcat-fedora/webapps] (err): Failed to generate additional resources using ‘eval_generate’: end of file reached

Sat Dec 19 22:58:21 -0500 2009 //Node[dds1]/Dds[dds]/Tomcat[dds]/File[/usr/local/tomcat-dds] (err): Failed to generate additional resources using ‘eval_generate’:

Tue Jan 05 17:57:50 -0500 2010 //Node[repo6]/Proai[proai]/Tomcat[proai]/File[/usr/local/tomcat-proai/webapps] (err): Failed to generate additional resources using ‘eval_generate’: Tue Jan 05 17:57:50 -0500 2010 //Node[repo6]/Proai[proai]/Tomcat[proai]/File[/usr/local/tomcat-proai/webapps] (err): Failed to retrieve current state of resource: Could not retrieve file metadata for puppet:///modules/tomcat/6.0.20/titles/proai/1.2/webapps: at /etc/puppet/production/modules/tomcat/manifests/init.pp:143

Wed Jan 06 07:41:48 -0500 2010 //wfi/Tomcat[wfi]/File[/usr/local/tomcat-wfi/webapps] (err): Failed to generate additional resources using ‘eval_generate’: end of file reached

Fri Jan 22 17:14:07 -0500 2010 //Node[repo1]/Fedora[fedora]/File[/usr/local/fedora] (err): Failed to generate additional resources using ‘eval_generate’: end of file reached

Fri Jan 22 17:53:22 -0500 2010 //Node[repo1]/Fedora[fedora]/File[/usr/local/fedora] (err): Failed to generate additional resources using ‘eval_generate’: end of file reached

Sat Jan 23 05:39:16 -0500 2010 //Node[repo1]/Fedora[fedora]/File[/usr/local/fedora] (err): Failed to generate additional resources using ‘eval_generate’: Could not intern_multiple from marshal: marshal data too short

Updated by James Turnbull almost 2 years ago

  • Target version changed from 0.25.5 to 49

Updated by S K over 1 year ago

  • Affected Puppet version changed from 0.25.4rc2 to 0.24.4

Hi everyone,

I have probably the same issue – getting about five false emails per day from various puppet hosts (triggered by various File resources):

Wed Jun 02 17:31:00 +0000 2010 //core::users::aduser/Module_file[/home/aduser/.ssh/adkey]/File[/home/aduser/.ssh/adkey] (err): Failed to generate additional resources using ‘eval_generate’:

running puppet 0.25.4 from EPEL on CentOS 5.4.

getting about five to ten messages per day from whole platform.

any ideas how to get rid of it?

thanks, Steve.

Updated by S K over 1 year ago

actually I can see the issue is related to the ‘source=>’ parameter of File resource. according to http://docs.puppetlabs.com/references/latest/type.html#source following syntax is supported

file { “/path/to/my/file”:

source => [
    "/modules/nfs/files/file.$host",
    "/modules/nfs/files/file.$operatingsystem",
    "/modules/nfs/files/file"
]

}

and the first existing source should be picked up. however this seems be the case of random failures (in my case) – in about 30% cases it sends false report complaining about some missing file with the suffix (eg. /modules/nfs/files/file.$host) instead of continuing in checking the next source – in my case there is always at least the file without any suffix (/modules/nfs/files/file) but the processing doesn’t get to this file.

To point out – this happens randomly in about 30% of runs. if the issue occurs and I restart the puppet agent to rerun again the issue doesn’t repeat. it is always related to the file {‘something": source = [“…/file.$host”, “…/file.$operatingsystem”, “…/file” ] }

Steve.

Updated by Markus Roberts over 1 year ago

  • Assignee deleted (Markus Roberts)
  • Target version changed from 49 to 4

Updated by James Turnbull 6 months ago

  • Target version deleted (4)

Also available in: Atom PDF