[Open.ogc] Deleted layers still exist?

Tue Sep 24 18:59:46 UTC 2013

Okay, I now feel a little silly. I did not replace the dummy password in my
notes file with my LDAP password. Shape files and zip files have the same
names as the layer. I attempted the data load again, using --update
overwrite, and I got a 201 response. However, when I look at the layer in
geoserver I see the old data. There is a column (property) called GEOID_jlf and
that column is in the old data layer and not in the new layer I'm
attempting to load.

[image: Inline image 1]

John F. Kennedy, PhD, CPG
 GIS Specialist
Contractor: Caelum Corp.
NOAA Fisheries Office of Science and Technology
1315 East-West Highway
SSMC3 Rm 9518
Silver Spring, MD 20910
301-427-8149

Comments? Have I been of assistance? Please let my supervisors know. :~)
Tim Haverland - NOAA Federal <Tim.Haverland at noaa.gov>
Hussain Jabalpurwala - NOAA Affiliate <hussain.jabalpurwala at noaa.gov>

On Tue, Sep 24, 2013 at 2:35 PM, Micah Wengren <micah.wengren at noaa.gov>wrote:

>  Ok.  Just curious, what are the shapefile names you're trying to
> upload?  Are they the communities_pts and communities_pgs layers?
>
> Looks like those are gone from the FeatureTypes list now (
> https://services.ogc.noaa.gov/geoserver/rest/workspaces/nmfs_st/datastores/postgis.html).
> But they still show up in the list when you go to the 'New Layer' page.
> Not sure I've seen that issue before, so might have to file a ticket.
>
> As for credentials, it should only be your NOAA LDAP username and
> password.  Those don't get passed on to the database, there's a separate
> way database authorization is handled.  If you can log in to the Web UI,
> there shouldn't be an issue with the REST API, or at least you shouldn't be
> getting that 401 response.
>
> Micah
>
>
> On 9/24/2013 2:21 PM, John Kennedy - NOAA Affiliate wrote:
>
> Thanks. Tomorrow works better for me. As for the credentials, nothing as
> changed on this end. I was able to load data with the hpcc_data.py script
> up to yesterday, but I did get an error message when attempting to
> overwrite the layer/feature type. Now, there maybe credential issues in the
> database (PostGIS?) but I've no way of knowing.
>
>  John F. Kennedy, PhD, CPG
>  GIS Specialist
>  Contractor: Caelum Corp.
> NOAA Fisheries Office of Science and Technology
> 1315 East-West Highway
> SSMC3 Rm 9518
> Silver Spring, MD 20910
> 301-427-8149
>
>  Comments? Have I been of assistance? Please let my supervisors know. :~)
> Tim Haverland - NOAA Federal <Tim.Haverland at noaa.gov>
>  Hussain Jabalpurwala - NOAA Affiliate <hussain.jabalpurwala at noaa.gov>
>
>
> On Tue, Sep 24, 2013 at 2:15 PM, Micah Wengren <micah.wengren at noaa.gov>wrote:
>
>>  Hi John,
>>
>> That is odd for the 401 response on the data upload.  Are you sure the
>> credentials are right?  The other two seem ok, depending on if the
>> featuretype existed in the first place or not.
>>
>> I might be able to come by later, but right now I have a deadline to
>> finish something this afternoon.  Let me know if you aren't able to get it
>> to work.
>>
>> Micah
>>
>>
>> On 9/24/2013 12:32 PM, John Kennedy - NOAA Affiliate wrote:
>>
>> Hi Micah,
>>
>>  Well, I think I'm getting close. I installed pip, python setup, and
>> request. I used hpcc_date_requests.py to delete the layer and the feature
>> type and I get these to responses:
>> Response: 200, url=
>> https://services.ogc.noaa.gov/geoserver/rest/workspaces/nmfs_st/datastores/postgis/featuretypes/communities_pgs
>>
>>  Response: 404, url=
>> https://services.ogc.noaa.gov/geoserver/rest/workspaces/nmfs_st/datastores/postgis/featuretypes/communities_pgs
>>
>>  The hpcc_date_requests appears to be attempting to load data, but the
>> response back is this:
>> Response: 401, url=
>> https://services.ogc.noaa.gov/geoserver/rest/workspaces/nmfs_st/datastores/postgis/file.shp?update=overwrite&configure=all
>>
>>  Any chance you can come to my cube and have a look?
>>
>>  Cheers,
>> John
>>
>>
>>
>>  John F. Kennedy, PhD, CPG
>>  GIS Specialist
>>  Contractor: Caelum Corp.
>> NOAA Fisheries Office of Science and Technology
>> 1315 East-West Highway
>> SSMC3 Rm 9518
>> Silver Spring, MD 20910
>> 301-427-8149
>>
>>  Comments? Have I been of assistance? Please let my supervisors know. :~)
>> Tim Haverland - NOAA Federal <Tim.Haverland at noaa.gov>
>>  Hussain Jabalpurwala - NOAA Affiliate <hussain.jabalpurwala at noaa.gov>
>>
>>
>> On Tue, Sep 24, 2013 at 10:18 AM, Micah Wengren <micah.wengren at noaa.gov>wrote:
>>
>>>  Hi John,
>>>
>>> How are you doing the attempted data upload?  With the python script?
>>> Can you tell me which python script you are using if so?  If you can, try
>>> using the hpcc_data_requests.py script instead of the other one (might
>>> involve some admin rights to install python modules on your workstation).
>>> I've update that one more recently.  I'd like to drop the other one at some
>>> point for easier maintenance.
>>>
>>> I think you should be able to upload a new data set to replace the
>>> existing one, even if it already exists in the database and data store (but
>>> could be wrong if there's an issue going on here with these layers).
>>> Something like the following should work:
>>>
>>> > hpcc_data_requests.py -a data -u <name> -p <pass> -w nmfs_st -f
>>> /path/to/shapefile.zip --type shape --update overwrite  (the shapefile
>>> would need to be named to match the layer you want to update).
>>>
>>>
>>> Also, this is something that is a little whacky with the GeoServer REST
>>> API (and should be better documented somewhere admittedly), but you may
>>> need to do a delete command for both the Layer and the FeatureType to fully
>>> delete it.  When you create a new layer with the Web UI, it actually makes
>>> both the Layer and FeatureType behind the scenes.  Not sure if this will
>>> fix this particular issue, but commands to do that would be something like:
>>>
>>> > hpcc_data_requests.py -a delete_layer -u <name> -p <pass> -w nmfs_st
>>> --layername <layername>
>>> > hpcc_data_requests.py -a delete_featuretype -u <name> -p <pass> -w
>>> nmfs_st --layername <layername>
>>>
>>> This should fully delete both a layer and its corresponding
>>> featuretype.  It's possible if you did a delete_layer command with via the
>>> script without doing delete_featuretype as well, some featuretypes got left
>>> behind.  Again, an issue that should really be better documented in
>>> GeoServer docs but unfortunately I haven't seen it described anywhere, I
>>> happened to find out myself after having some similar issues.
>>>
>>> I'm looking around the workspace now, and I think I see some cases where
>>> there are FeatureTypes that aren't associated with any layers, in
>>> particular 'communities_pgs' and 'communities_pts' are both FeatureTypes
>>> without corresponding layers.  This could be preventing the data upload
>>> from happening for these layers.  I'm just looking at these two lists:
>>>
>>>
>>> https://services.ogc.noaa.gov/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layer.LayerPage(filtered by 'nmfs_st')
>>>
>>> https://services.ogc.noaa.gov/geoserver/rest/workspaces/nmfs_st/datastores/postgis.html
>>>
>>> Also, you should probably do the delete_featuretype command for
>>> 'communities_pgs20130506' and 'communities_pts20130506', since they don't
>>> have corresponding Layers.  I think that will get those two in sync.
>>>
>>> I'm not sure why you get such a long list in the New Layer dialog
>>> though, looks like a bug to me.  No idea where 'SI_SE_Gulf_sites_pg_ALLsites'
>>> and 'SI_SE_Gulf_sites_pt' are coming from, because those aren't listed
>>> as current FeatureTypes in the REST API.
>>>
>>> HTH,
>>> Micah
>>>
>>>
>>>
>>>
>>>
>>> On 9/24/2013 9:30 AM, John Kennedy - NOAA Affiliate wrote:
>>>
>>> The two layers of concern are communities_pgs and communities_pts. When
>>> I attempt to load new data, via a zip file with a shape file, I get a
>>> message that the communities_pgs exists. When I look at the list of source
>>> under the "Add sources" button I can see the communities_pgs layer and the
>>> publish link. But when I click the link and attempt to save I get the
>>> message below. Deleting layer via the hpcc_data python script also seems to
>>> fail.
>>>
>>>  [image: Inline image 1]
>>>
>>>  John F. Kennedy, PhD, CPG
>>>  GIS Specialist
>>>  Contractor: Caelum Corp.
>>> NOAA Fisheries Office of Science and Technology
>>> 1315 East-West Highway
>>> SSMC3 Rm 9518
>>> Silver Spring, MD 20910
>>> 301-427-8149
>>>
>>>  Comments? Have I been of assistance? Please let my supervisors know.
>>> :~)
>>> Tim Haverland - NOAA Federal <Tim.Haverland at noaa.gov>
>>>  Hussain Jabalpurwala - NOAA Affiliate <hussain.jabalpurwala at noaa.gov>
>>>
>>>
>>> On Tue, Sep 24, 2013 at 9:20 AM, Tim Haverland - NOAA Federal <
>>> tim.haverland at noaa.gov> wrote:
>>>
>>>> I'm adding John Kennedy to this thread so he can work with you on this
>>>> issue.
>>>>
>>>>  Tim
>>>>
>>>>
>>>> On Tue, Sep 24, 2013 at 8:49 AM, Micah Wengren <micah.wengren at noaa.gov>wrote:
>>>>
>>>>>  Tim,
>>>>>
>>>>> I can see some layers in your workspace in the New Layer dialog that
>>>>> aren't currently published and have a 'Publish' link next to them, so I see
>>>>> what you mean.  Are you able to create a new SQL view layer with the same
>>>>> name as one of those layers, or does that not work?  Can you give me the
>>>>> name of one of the layers that you've tried with and I'll pass it on to
>>>>> OpenGeo support to troubleshoot, that way I can give them a specific layer
>>>>> name with the issue.
>>>>>
>>>>> The layers should be disappearing but for whatever reason they're
>>>>> not.
>>>>>
>>>>> Micah
>>>>>
>>>>>
>>>>>
>>>>> On 9/23/2013 3:55 PM, Tim Haverland - NOAA Federal wrote:
>>>>>
>>>>>  Hi all,
>>>>>
>>>>>  We've been trying to clean up a bunch of layers in our workspace. We
>>>>> can delete layers using the "Remove selected resources" button on the
>>>>> Layers page, but when we try to replace that layer with another, the layer
>>>>> is still there.
>>>>>
>>>>>  We can verify this by clicking "Add a new resource" ... all of the
>>>>> old layers are still there and I can click Publish to make them "live"
>>>>> again.
>>>>>
>>>>>  How do we permanently get rid of unwanted layers?
>>>>>
>>>>>  Tim
>>>>> --
>>>>> *Tim Haverland*
>>>>> Acting Operations Branch Chief
>>>>> NOAA Fisheries Office of Science and Technology
>>>>> 1315 East-West Highway
>>>>> SSMC3 Rm 12303
>>>>> Silver Spring, MD 20910
>>>>> 301-427-8137
>>>>>
>>>>>
>>>>>  _______________________________________________
>>>>> Open.ogc mailing listOpen.ogc at list.woc.noaa.govhttps://list.woc.noaa.gov/cgi-bin/mailman/listinfo/open.ogc
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>  --
>>>> *Tim Haverland*
>>>> Acting Operations Branch Chief
>>>> NOAA Fisheries Office of Science and Technology
>>>> 1315 East-West Highway
>>>> SSMC3 Rm 12303
>>>> Silver Spring, MD 20910
>>>> 301-427-8137
>>>>
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://list.woc.noaa.gov/pipermail/open.ogc/attachments/20130924/07e1f685/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 164721 bytes
Desc: not available
URL: <https://list.woc.noaa.gov/pipermail/open.ogc/attachments/20130924/07e1f685/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 153411 bytes
Desc: not available
URL: <https://list.woc.noaa.gov/pipermail/open.ogc/attachments/20130924/07e1f685/attachment-0003.png>