New task was created

Task was assigned to Taina Wanderley

Status change: Open → In Progress

So... just to confirm "piece by piece" means to load this data for each city already activated here? For ex, I got to Geneva, "stocks and flows" and in "Extraction: Agriculture" I put this data from FAO for Switzerland.
The CSV format is good, right?

ok, this is part of the multi-scale integration that we're talking about. So, maybe there will be a country-level sub-site?

Hi Taina,

In this case piece-by-piece refers to each of the flows (extraction -- imports -- exports -- etc). Each of them needs to be uploaded separately instead of us being able to load an entire MFA dataset that contains all information at once (the IRP dataset, which unfortunately does not come with an open license).

So you can load ALL info for extraction that is available at once. Where exactly to put it we can discuss later today in the sprint.

Tks Paul!
We'll discuss later then.
see you soon

Hi Taina, just to recap from our meeting: you can use Munich as the place to dump ALL data for multiple scales such as global datasets on all cities or countries.

Secondly, please note I have activated Brasília here.

Hey, Paul, thank you! I'll work on it :)

Hi, Paul!
I'm progressing in this task, with this 2 uploads completed:

FAOSTAT Crops Data of all countries
FAOSTAT Forestry Production and Trade Data of all countries

Everything is being registered in the city of Munich, as agreed. As I'm having some issues with the files, I ask that if anyone wants to check it out to see if everything is alright.... from what I've seen, these first two items are :)

BUT, when I start a third item "FAOSTAT Detailed trade matrix Data (imports/exports) of all countries", I got the following error " 413 Request Entity Too Large" and I couldn't load the files here.
I tried a few things but nothing worked.
Any idea how I do it, Paul?

Hi Taina,

Great work already! I will check those out over the weekend and let's chat on Monday's sprint if you are there. For the error that you get... how large is the file you are trying to upload? Sounds like the file size is an issue.

okay, yes, I'll be in the meeting next Monday; if I can not do it, I'll let you know :)
Yes, this is clearly a file size issue haha ; 1,32 GB
Maybe for this one I could upload the zip file? (I didn't do it, because I remember you saying to not)

Last edited: 2021-06-18 17:57:53.698311+00:00

2 more complete uploads:
FAOSTAT Livestock Data of all countries
FAOSTAT Agriculture Total Emissions Data of all countries

I have to change the license of all FAO itens to: "CC BY-NC- SA 3.0 IGO" , according to the document

Last edited: 2021-06-21 12:48:04.416775+00:00

Hi Taina, we didn't catch you in the call but hereby some replies:

For the large file, can you find out one thing: is it possible to generate a URL for the particular dataset? Currently the link goes to the general FAO data portal... is there a way to simply link directly to the 1.32 GB large file in the URL field? That would also help me investigate the file structure itself.

With the newly added documents: great work!! Can I also ask you to add links to these files in the multi-level specification so that everyone is aware they have been added?

With the specific license: good you found out about this. That specific license was not yet available before in MOC, but I have now added it. Can you change the items in the libraries to this specific license?

Thanks!

Last edited: 2021-06-21 15:33:42.477574+00:00

Hi Paul!
first, I'm sorry about not being in the call today, I couldn't do it this time :/
Anyway, thank you for your replies.

For the large file, yes, you can find the specific dataset here in "Bulk Downloads" > "all data" .
So I'll update all the files I uploaded with the correct license (I saw that we didn't have this one yet) and the specific links.
And, yes, I can also add links to these files in the multi-level specification.

Noted, thanks Taina.
In terms of ALL DATA, when I try and open it, it links to this file. However, that file is 60 MB. Could you pinpoint the actual link to make sure we are talking about the same file? Thanks!

Hi Paul!
Sorry, I don't know where I saw a file with 1.32 GB (maybe just in my imagination)
Within the 60MB zip file there are: a 287 KB file, a 220 KB file and a 1KB file (I imagine you already know haha)
But still I can't upload! Don't know why

Status change: In Progress → On Hold