NoneType' object has no attribute 'items' #56

gthomson31 · 2019-09-17T07:35:13Z

Hi There

Stumbled upon this issue when building out a flask application but have also noticed this when just testing the code in a simple consistent loop.

Looking for pointers if there is something obvious I am missing in this.

Using Username/Password to authenticate.

Error usually occurs after 10-15 mins of running without issue.

SCRIPT CODE

foundation_name = 'lab'
target_endpoint = '<URL>'

proxy = dict(http=os.environ.get('HTTP_PROXY', ''), https=os.environ.get('HTTPS_PROXY', ''))
client = CloudFoundryClient(target_endpoint, proxy=proxy, verify=False)

client.init_with_user_credentials('<USERNAME>','<PAASWORD>')

def app_buildpacks():
	app_buildpack_usage = []
	app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename' , 'Locked']

	
	for app in client.v2.apps:
		app_name = app['entity']['name']
		app_buildpack_name = app['entity']['detected_buildpack'] or app['entity']['buildpack'] 
		app_buildpack_guid = app['entity']['detected_buildpack_guid']


		for buildpack in client.v2.buildpacks:

			buildpack_guid = buildpack['metadata']['guid']

			if app_buildpack_guid == buildpack_guid:

				buildpack_name = buildpack['entity']['name']
				buildpack_stack = buildpack['entity']['stack']
				buildpack_filename = buildpack['entity']['filename']
				buildpack_lock = buildpack['entity']['locked']

				d = { 
				'app_name' : app_name , 
				'buildpack_name' : buildpack_name,
				'buildpack_stack' : buildpack_stack , 
				'buildpack_guid' : buildpack_guid, 
				'buildpack_filename': buildpack_filename,
				'buildpack_lock' : buildpack_lock
				}

				app_buildpack_usage.append(d)

	# Sort the list by name 
	app_buildpack_usage = sorted(app_buildpack_usage, key = lambda i: i['buildpack_name'])
	
	# Return the two values ( Builpack usage list of dicts and table headers )
	return app_buildpack_usage,app_buildpack_table_field_names


def background():
  global app_buildpack_usage, app_buildpack_table_field_names

  while True:
    try:
        app_buildpack_usage,app_buildpack_table_field_names = app_buildpacks()



def foreground():
  @app.route('/app_buildpacks')
  def app_buildpacks_page():
    return render_template('app_buildpack_table.html', colnames=app_buildpack_table_field_names, app_buildpack_usage=app_buildpack_usage)



b = threading.Thread(name='background', target=background)
f = threading.Thread(name='foreground', target=foreground)


b.start()
f.start()

ERROR

  \Python37\lib\site-packages\cloudfoundry_client\v2\entities.py", line 66, in _list
    yield entity_builder(list(resource.items()))

  \Python37\lib\site-packages\cloudfoundry_client\v2\apps.py", line 51, in <lambda>
    lambda pairs: _Application(target_endpoint, client, pairs))

  \Python37\lib\site-packages\cloudfoundry_client\v2\entities.py", line 17, in __init__
    for attribute, value in list(self['entity'].items()):

AttributeError: 'NoneType' object has no attribute 'items'

The text was updated successfully, but these errors were encountered:

antechrestos · 2019-09-24T14:08:45Z

Hi,

I've been running the following code and it works nice... 😢

import logging
import os
import threading

from cloudfoundry_client.client import CloudFoundryClient
from flask import jsonify, Flask

logging.basicConfig(level=logging.INFO,
                    format='%(levelname)5s - %(name)s -  %(message)s')

foundation_name = 'lab'
target_endpoint = 'https://somewhere.org'

proxy = dict(http=os.environ.get('HTTP_PROXY', ''), https=os.environ.get('HTTPS_PROXY', ''))
client = CloudFoundryClient(target_endpoint, proxy=proxy, verify=False)

client.init_with_user_credentials('user', 'password')

app_buildpack_usage = []
app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename', 'Locked']

def app_buildpacks():
    app_buildpack_usage = []
    app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename', 'Locked']

    for app in client.v2.apps:
        app_name = app['entity']['name']
        app_buildpack_name = app['entity']['detected_buildpack'] or app['entity']['buildpack']
        app_buildpack_guid = app['entity']['detected_buildpack_guid']

        for buildpack in client.v2.buildpacks:

            buildpack_guid = buildpack['metadata']['guid']

            if app_buildpack_guid == buildpack_guid:
                buildpack_name = buildpack['entity']['name']
                buildpack_stack = buildpack['entity']['stack']
                buildpack_filename = buildpack['entity']['filename']
                buildpack_lock = buildpack['entity']['locked']

                d = {
                    'app_name': app_name,
                    'buildpack_name': buildpack_name,
                    'buildpack_stack': buildpack_stack,
                    'buildpack_guid': buildpack_guid,
                    'buildpack_filename': buildpack_filename,
                    'buildpack_lock': buildpack_lock
                }

                app_buildpack_usage.append(d)

    # Sort the list by name
    app_buildpack_usage = sorted(app_buildpack_usage, key=lambda i: i['buildpack_name'])

    # Return the two values ( Builpack usage list of dicts and table headers )
    return app_buildpack_usage, app_buildpack_table_field_names


def background():
    global app_buildpack_usage, app_buildpack_table_field_names

    while True:
        try:
            app_buildpack_usage, app_buildpack_table_field_names = app_buildpacks()
            logging.info("App buildpacks refreshed")
        except Exception as ex:
            logging.exception("Exception while getting buildpacks")


def foreground():
    global foundation_name
    app = Flask(foundation_name)

    @app.route('/app_buildpacks')
    def app_buildpacks_page():
        # return render_template('app_buildpack_table.html', colnames=app_buildpack_table_field_names,
        #                        app_buildpack_usage=app_buildpack_usage)
        return jsonify(app_buildpack_usage)

    app.run(port=os.environ.get('PORT', 9010))


b = threading.Thread(name='background', target=background)
f = threading.Thread(name='foreground', target=foreground)

b.start()
f.run()

I will publish a version that improve the information got from error and will let you use it to better en light it

Thanks

antechrestos · 2019-09-24T14:12:43Z

For infomormation I also rewrote a little your code as follows to optimize api call:

import logging
import os
import threading

from cloudfoundry_client.client import CloudFoundryClient
from flask import Flask, jsonify

_logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO,
                    format='%(levelname)5s - %(name)s -  %(message)s')

target_endpoint = 'https://somewhere.org'

proxy = dict(http=os.environ.get('HTTP_PROXY', ''), https=os.environ.get('HTTPS_PROXY', ''))
client = CloudFoundryClient(target_endpoint, proxy=proxy, verify=False)

app = Flask(__name__)

app_buildpack_usage = []

app_buildpack_table_field_names = ['App Name', 'Buildpack Used', ' Buildpack Stack', 'GUID', 'Filename', 'Locked']


def app_buildpacks():
    global _logger

    next_app_buildpack_usage = []

    buildpacks_by_guid = {buildpack['metadata']['guid']: buildpack for buildpack in client.v2.buildpacks}

    for app in client.v2.apps:
        app_name = app['entity']['name']
        app_buildpack_guid = app['entity']['detected_buildpack_guid']
        app_buildpack = buildpacks_by_guid.get(app_buildpack_guid)
        if app_buildpack is not None:
            buildpack_name = app_buildpack['entity']['name']
            buildpack_stack = app_buildpack['entity']['stack']
            buildpack_filename = app_buildpack['entity']['filename']
            buildpack_lock = app_buildpack['entity']['locked']

            d = {
                'app_name': app_name,
                'buildpack_name': buildpack_name,
                'buildpack_stack': buildpack_stack,
                'buildpack_guid': app_buildpack_guid,
                'buildpack_filename': buildpack_filename,
                'buildpack_lock': buildpack_lock
            }

            next_app_buildpack_usage.append(d)

    # Sort the list by name
    return sorted(next_app_buildpack_usage, key=lambda i: i['buildpack_name'])


def background():
    global app_buildpack_usage, _logger

    while True:
        try:
            app_buildpack_usage = app_buildpacks()
            _logger.info('App buildpacks refreshed')
        except Exception as _:
            _logger.exception("Error while refreshing buildpacks")
            return


@app.route('/app_buildpacks')
def app_buildpacks_page():
    global app_buildpack_table_field_names, app_buildpack_usage
    # return render_template('app_buildpack_table.html', colnames=app_buildpack_table_field_names,
    #                        app_buildpack_usage=app_buildpack_usage)
    return jsonify(app_buildpack_usage)


background_thread = threading.Thread(name='background', target=background)
if __name__ == '__main__':
    client.init_with_user_credentials('user', 'password')
    background_thread.start()
    app.run(port=os.environ.get('PORT', 9000))

It may be good to add a sleepin your background so as not to flaod your endpoint

See #56

antechrestos · 2019-09-27T12:30:03Z

@gthomson31 I've just released the version 1.9.0 that will raise a dedicated exception InvalidEntityException containing the raw entity. Can you please run your code with this version and send me the error.

Thanks

gthomson31 · 2019-10-01T13:43:22Z

Cheers will give it a go

kcheng555 · 2019-12-09T18:31:29Z

Hello there,
Has any progress been made on this? Also, I'm using version 1.9.0 it seems that this issue is still raising an AttributeError.

  File "/usr/lib/python3.7/site-packages/cloudfoundry_client/v2/entities.py", line 70, in _list
     yield entity_builder(list(resource.items()))
  File "/usr/lib/python3.7/site-packages/cloudfoundry_client/v2/apps.py", line 51, in <lambda>
     lambda pairs: _Application(target_endpoint, client, pairs))
  File "/usr/lib/python3.7/site-packages/cloudfoundry_client/v2/entities.py", line 21, in __init__
     for attribute, value in list(self['entity'].items()):
AttributeError: 'NoneType' object has no attribute 'items'

DerCoop · 2019-12-09T20:25:12Z

Hey kcheng55, thx for the report. please update your cloudfoundry client to the 1.10.

are your sure that you use the code of 1.09? The exception was fixed exactly with this version but your code looks like the older one.

kcheng555 · 2019-12-09T20:32:01Z

I'm working on updating to 1.10.0 to see if that will fix the issue, but I'm not sure how to reproduce it, so it's a waiting game.

I've verified that I'm using version 1.9.0 by printing cloudfoundry_client.__version__. It looks like my stack trace has slightly different line numbers than the one in the OP, which seem to line up with the changes made here : 435d109

antechrestos · 2019-12-09T23:39:28Z

That seems to be related to entity key being present in json object yet valued to null
I am currently on holiday without any laptop

DerCoop · 2019-12-10T06:22:01Z

@antechrestos yes, but you fixed (add a check) it with the 1.9 ;) So, happy hollliday ;)

antechrestos · 2019-12-10T10:09:32Z

Partially it seems. I though the entity was not present. In fact it is but valued to null. If that is the case, what to do with it?

DerCoop · 2019-12-10T10:29:30Z

null object is, like no object, not parseable. In both cases we will raise an invalid entity so

extend the test to check if the field exists and is not null
something like
if self.get('entity') is not None

antechrestos · 2019-12-10T10:32:19Z

Agreed

This change enforce type checking for entity attribute in object returned See #56

antechrestos pushed a commit that referenced this issue Sep 27, 2019

Raise 'InvalidEntityException' if entity is malformed

435d109

See #56

antechrestos added a commit that referenced this issue Dec 28, 2019

Enforce entity attribute check

4c5023e

This change enforce type checking for entity attribute in object returned See #56

antechrestos mentioned this issue Dec 28, 2019

Enforce entity attribute check #70

Merged

antechrestos added a commit that referenced this issue Dec 28, 2019

Enforce entity attribute check

ee74a94

This change enforce type checking for entity attribute in object returned See #56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NoneType' object has no attribute 'items' #56

NoneType' object has no attribute 'items' #56

gthomson31 commented Sep 17, 2019 •

edited

antechrestos commented Sep 24, 2019

antechrestos commented Sep 24, 2019

antechrestos commented Sep 27, 2019

gthomson31 commented Oct 1, 2019

kcheng555 commented Dec 9, 2019

DerCoop commented Dec 9, 2019

kcheng555 commented Dec 9, 2019

antechrestos commented Dec 9, 2019

DerCoop commented Dec 10, 2019

antechrestos commented Dec 10, 2019

DerCoop commented Dec 10, 2019

antechrestos commented Dec 10, 2019

NoneType' object has no attribute 'items' #56

NoneType' object has no attribute 'items' #56

Comments

gthomson31 commented Sep 17, 2019 • edited

antechrestos commented Sep 24, 2019

antechrestos commented Sep 24, 2019

antechrestos commented Sep 27, 2019

gthomson31 commented Oct 1, 2019

kcheng555 commented Dec 9, 2019

DerCoop commented Dec 9, 2019

kcheng555 commented Dec 9, 2019

antechrestos commented Dec 9, 2019

DerCoop commented Dec 10, 2019

antechrestos commented Dec 10, 2019

DerCoop commented Dec 10, 2019

antechrestos commented Dec 10, 2019

gthomson31 commented Sep 17, 2019 •

edited