Archive Insights - export all your Insights data to a NAS, FTP or web folder

[Gruijter]

Archive Insights

Export Insights

Make a backup of all your insights to your NAS, FTP or WebDAV compliant storage. The insights are zipped and stored in .csv format so you can open them in excel and create all kinds of statistics and charts. The export also includes a JSON version of the data.

Backups can be easily scheduled by using the trigger card in a flow. You can schedule a full backup, or a backup per app.

To setup enter the local share (NAS) information and/or the webDAV information. It is possible to use a local share, FTP and a webDAV storage simultaneously.

Note: making a backup can take a long time (more than 10 minutes), depending on how many insights you have. It is not possible to restore a backup into Homey.

Donate:

If you like the app do not hesitate to donate a cool drink

===============================================================================

Version changelog: changelog.txt

[Gruijter]

[Gruijter]

I just uploaded the new Homey V2 compatible version of the Export Insights app to the appstore. After approval it will be available in the beta stream. The stable release will still be for Homey V1, but will eventually be deprecated.

The new version allows you to store all Insights, or Insights of a specific app, to a network drive (NAS) or webDAV folder. It also includes Insights that you cannot get from Athoms Insights page!

Please give me some feedback, so I can improve it even more before making it stable.

[Gruijter]

The version 2.2.0 was just approved in the appstore, so you can start installing NOW. Note: it is the beta version

[Camelen]

I tried it and it manages to export my “smaller” apps but gets an error on Fibaro devices. Not sure if it’s the specific app or if it’s because my vast majority of devices are fibaro.

Edit: It would be awesome to see if it could create a folder with incrementing numbers to put the files in. Say I want to export once every week and it puts it in a folder called 2019 and in a subfolder by week number or month or something

2019-03-28 20:46:08 [err] [App] unhandledRejection! { rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument: rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument at /node_modules/athom-api/dist/index.js:1:1076029 at at process._tickCallback (internal/process/next_tick.js:189:7) name: ‘rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument’, code: 500, cause: t { __athom_api_type: ‘HomeyAPI.ManagerInsights.Error’, code: 500, error: ‘rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument’, error_description: ‘rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument’, ‘$stack’: undefined } }
2019-03-28 20:45:49 [log] [App] getting logs related to com.fibaro

[Gruijter]

Hi John,

Thx for trying/testing the new app version! Concerning the error, could you please make an issue for that on Github? Since I cannot reproduce this error myself, I have some more questions for you https://github.com/gruijter/com.gruijter.insights2csv/issues

Creating subfolders would make my app rather complex.I would somehow have to keep track of the previously created subfolders (and for each user that would be different). Or do you have another idea? And can you explain how this folder structure would be helpful to you? Maybe you can explain more how you want to use the exported data?

Would it help you if I move the creation date/time of the zips to the beginning of the filename? That way it is easy to sort all the files based on date (actually you can already do that now based on the file creation date).

cheers

[Camelen]

I’ll try to make an issue on GitHub. I’ve got no user there yet but I’ve been lurking a bit

Regarding the folders, it’s no big deal. If it’s hard to implement maybe it was just a bad idea. It’s pretty easy to move things myself now and then.

At the moment my plan is to keep statistics on temperature and humidity on the outside and on 2 different places in my house (in the trusses). I will use the info to see if the conditions are good enough to prevent damage to the house at different parts of the year and see if things improve by taking different kinds of measures and comparing the data from the previoud year for example.

There will be a lot of data and automatically sorting the data in folders by date taken for example would help. And I was thinking folders since there were quite some files that got created each time so the folder it gets saved to will look quite busy in a short amount of time.

Like I said though it’s not a very big deal and can easily be done manually

[Gruijter]

Insights export

Thx… but why a Nas , why not email ? Can’t see it can get info from one device. why only .csv format, .PDF is a good format. /Anders

Hi @Anders_P_Jensen, let’s continue the discussion here.

The goal of my app is to export the insights as data, not as a graph. But if you have another use case, please share what you need and why. Maybe we can find a solution together

[Max_de_V]

Hi Gruijter, thanks for contributing!
I have installed your app, but I do not have a NAS or something else. I am looking for a way to download to a folder on the computer. Is that also possible? Where can I do the setup?

Thank you.
Max

[Gruijter]

Yes, you can make a network share on your desktop computer. It then works the same as for a NAS. Just make sure your desktop/laptop is on and on the network when you start an export.

For Windows 10 you can find tutorials how to make a network share, e.g. this one: https://www.tenforums.com/tutorials/111783-share-files-folders-over-network-windows-10-a.html

[Gruijter]

Hi @Camelen ,

I just finished v2.3.0 with significant stability changes. Can you check if that solves your issue with exporting Fibaro? v2.3.0 is now on Github, and will be released in the appstore as soon as Athom approves it.

[Camelen]

Great! I will try it as soon as they approve it and report back

[Gruijter]

V2.3.0 was just released in the appstore. It is now the stable version. Most important change is that I added internal queueing for export jobs. This reduces peak cpu and memory load on Homey.

This version only works on Homey V2. Users that are still on V1 can keep using the old app, but it is not possible to install it anymore from the appstore.

[Camelen]

Difference from before seems to be that it doesn’t stop when the error with fibaro comes but continues after and finishes the rest of the exports.

Same error seems to exist with the fibaro app for me.

2019-04-16 12:30:41 [err] [App] { rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument: rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument at /node_modules/athom-api/dist/index.js:1:1076029 at at process._tickCallback (internal/process/next_tick.js:189:7) name: ‘rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument’, code: 500, cause: t { __athom_api_type: ‘HomeyAPI.ManagerInsights.Error’, code: 500, error: ‘rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument’, error_description: ‘rrdcached: rrd_fetch_r failed: mmaping file ‘/opt/user/manager/insights/8e4356272fd93eaadd7d86454c5d2137.rrd’: Invalid argument’, ‘$stack’: undefined } }
2019-04-16 12:30:22 [log] [App] getting logs related to com.fibaro

[Gruijter]

Could you make an issue on Github for this error? I have some more questions about it. Thx!

[Camelen]

I’ll try to find the time to do this.

[Gruijter]

New version was just released in the appstore. @Camelen: Could you test to see if this version fixes your issue?

Have fun all!

[Camelen]

Hey!

Thank you for trying to fix it. I know that I’m not as helpful as I can be and can’t really expect it to be fixed until I commit myself a bit more to it.

But I tried yesterday and my problem still persists.

[OttoWerse]

Is there any explanation on how to set this up properly anywhere? I have already read the description in the homey App store and on GitHub, but I can’t seem to find any information whatsoever, so I can’t figure out how to do anything with this app.

It won’t find my NAS, I have no idea why. All I get is a cryptic error message. Error: getaddrinfo ENOTFOUND followed by whatever I put in “Network Share” and the number 445. (I guess it means Port 445?) I’ve already forwarded Port 445 on the machine and my router. (even though this seems like a terrible idea, from a security standpoint, but whatever) but that didn’t change anything.

Is there anything else on the Server side that needs to be set up? What formatting is required for the NETWORK SHARE (which I can only guess is for the server’s network internal address)?
How do I use drive letters in the file path?
What is the use of the DOMAIN field (I’ve never seen this field in the context of a local NAS setup)?

This is, by far, not my first Networking rodeo, but at least some documentation would be helpful here.

[robertklep]

I don’t run this app, so I have to go on what the documentation and the first post in this thread states, but:

• you need to enable SMB/Samba on your NAS
• the network share should be //HOSTNAME/SHARENAME; for instance, my Synology NAS is available as //syno (it says so in the control panel where you enable SMB, “syno” is the hostname) and I can attach the name of a shared folder: //syno/InsightsBackup
• DOMAIN is an SMB setting. On my Synology, the domain is WORKGROUP (again, that’s shown in the control panel)
• I don’t know why you would need to add a port number, SMB has a default port
• getaddrinfo means that Homey cannot resolve the hostname of your NAS; instead of using a textual hostname, you could try to use an IP-address. In my case, that would be //192.168.23.13/InsightsBackup