•   almost 5 years ago

Geo map using BigSheets hangs

Hi

I have uploaded a data set to HDFS and then created a worksheet in BigSheets. I was able to do a simple bar chart using that.

When I try to create a Geo Map using the BigSheets "Add Chart" feature, it seems to hang, I have left it running for more than 10 minutes and still see the spinning wheel. The data set I use has latitude and longitude values which I feed to the Geo Map template provided by BigSheets.

The data set I have is about 2 million records, so unsure if that is what takes so much time. The "Application Status" tab on the BigInsights web console does not show any active/running jobs and no failed jobs either.

Has any one tried this and if so, have you been able to create a Map-based chart successfully? Do share your thoughts.

Regards
Vinayak

  • 5 comments

  •   •   almost 5 years ago

    Hi Vinayak,

    I tried the same thing and it worked for me, but only after I enabled browser plugins. The Geo Map is Flash based, but my Chrome browser was blocking the Flash content from BlueMix. Is it possible this is the same issue you encountered? Using Chrome I was able to re-enable the plugin for the BlueMix domain by clicking the puzzle-piece looking icon for plugins on the far right of the address bar and then selecting "Always allow plug-ins on ".

    If this isn't the same issue that you encountered can you please say which browser version you are using? Also, if you are certain that Flash is enabled for your browser would you be willing to provide all the values for the chart settings that you used, as well as a small snippet of the data for only the LAT and LON columns? Even just the format of these values alone might give some clue about whether this issue is related to LAT LON formatting. Also, can you say whether or not there are null LAT and LON vals in the data set? That might also be a factor.

    Below is some made up test data that worked for me after I created a workbook using the TSV reader. I know the map accepts LAT and LON in the format shown. btw, in my case, it doesn't show much for a density style map because I only have four data points. For you with 2 million you should at first see a very small portion of the map displaying with some content, but then after you run the map, which runs a map reduce job, you should see the density shown for the entire map properly.

    This was the data I used to test:

    CITY LAT LON POP
    cityA 35.929673 -78.948237 11
    cityB 38.889510 -77.032000 22
    cityC 38.032120 -78.477510 33
    cityD 36.379450 -75.830290 44

    Thanks,

    Kevin

  •   •   almost 5 years ago

    Hi Kevin

    Thanks for pointing out the browser plugins, I did check that and it seems that the Adobe Flash Player plugin is enabled. Listed below is the version

    Adobe Flash Player (2 files) - Version: 15.0.0.239
    Shockwave Flash 15.0 r0

    I am using the Chrome Browser Version 39.0.2171.71 m.

    After getting your comment, I tried the same set of tasks with a smaller data set (top 10K records) and it works i.e. I do get a US map, with some density highlighted. So unsure if there is a data issue with the rest of the file. I have ensured locally (on my PC) that there are no null or NA values. I tried a Geo Map as well as a regular Map and both "hang" with the larger data set.

    Also, since I can see the chart with the smaller (10K) data set, I was wondering if features such as zoom and pan exist, do you know? I don't see any control to do that and the default double-click or mouse-wheel scroll do not seem to do anything. When I click on a US state it gets highlighted but unable to "zoom" into it to lower levels such as cities and zip codes. Have you done this or could you point me to some documentation as to how to get this?

    Thanks for your input.

    Regards
    - Vinayak

  •   •   almost 5 years ago

    Hi Vinayak,

    Sorry, the maps do not have any zoom capability. It is limited currently to only showing either the world or US map as it is.

    I wonder if you are hitting a new issue which hasn't been encountered in testing, due to the number of points being plotted. We will have to investigate on our end, but if we find the cause it might not be patched in time for the challenge to complete. As a work-around, if you need to use the chart maybe the chart will be able to handle enough points matching only to specific regions (i.e. LAT > X, LON < Y). For instance, if you're using the world map, then maybe try plotting only the North American points in a North America map, for instance.

    If I do confirm there is a bug in the map due to data volume being charted, rather than any such data formatting issue, I'll reply again to this post.

    Thanks,

    Kevin

  •   •   almost 5 years ago

    Hi Kevin,

    I have a question about the map. I have the same issue of Vinayak: I tried to show 300K of points (with Latitude and Longitude values) and, if the Flash Player does not crash, I can see the points on the map.
    I will try to reduce the number of geographic points (from 300K to 20K).
    The main issue is that all the informations only regard London city so, in the world map, I can see only 1 little point in proximity of London.
    It is possible to add a UK map or Europe Map?

    Regards,
    Michele

  •   •   almost 5 years ago

    Hello Michele,

    We don't have any maps that are of a finer granularity out of the box. And the one we do have is not able to display an unbounded number of points (as you and Vinayak have seen).

    You may not want to take the time and effort to make your own chart, but it is possible. We have a way of extending a BigInsights instance with new charts but we do not have any examples posted for how to extend them. BigSheets can be extended with new readers, new functions and new charts, but we only expose Eclipse tooling to make it easier to build the new readers and new functions but not yet the charts.

    Without an example it may be quite hard to extend the interface, but you are welcome to try to build your own chart by referencing the BigSheets plug-in documentation here: http://www-01.ibm.com/support/knowledgecenter/#!/SSPT3X_3.0.0/com.ibm.swg.im.infosphere.biginsights.analyze.doc/doc/r0057529.html

    Thanks,

    Kevin

Comments are closed.