Error when invoke endpoint

[cnasri]

Hello,

I
When I try to invoke the endpoint and start the prediction (I use the same aws code of aws) , I get this error below

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from model with message "

<title>500 Internal Server Error</title> Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
". See https://eu-west-1.console.aws.amazon.com/cloudwatch/home?region=eu-west-1#logEventViewer:group=/aws/sagemaker/Endpoints/sagemaker-all-2018-04-11-12-00-21-560 in account 180856571690 for more information.

I don't think that the problem come from instance type because i am using a small dataset using decision trees.
Can you help me to resolve this problem?
Thanks

[djarpin]

Hi @cnasri - Were there additional details in the CloudWatch logs that the above error links to (Note, you may have to filter through the GET /ping lines to find the error)? Thanks.

[Schmidtbit]

Hi @djarpin, when I go to my CloudWatch logs I can't see anything. I went to CloudWatch > Log Groups > my endpoint and there are just a bunch of "inference data loaded successfully" logs. How do I find the GET /ping lines?

[Schmidtbit]

@cnasri did you ever resolve this issue? I am having same problems:

try:
    from StringIO import StringIO
except ImportError:
    from io import StringIO

from io import BytesIO
import csv
import sys, os, base64, datetime, hashlib, hmac
from chalice import Chalice, NotFoundError, BadRequestError
import boto3


app = Chalice(app_name='<app-name>')
app.debug = True

sagemaker = boto3.client('sagemaker-runtime')

@app.route('/', methods=['POST'], content_types=['application/x-www-form-urlencoded'])
def handle_data():
    input_text = app.current_request.raw_body
    print(input_text)
    res = sagemaker.invoke_endpoint(
                    EndpointName='<my-endpoint>',
                    Body=input_text,
                    ContentType='text/csv',
                    Accept='Accept'
                )
    return res['Body'].read().decode()[0]

[ZMarouani]

Was this issue resolved ? would this be a permissions problem ?

[kaddynator]

I was able to resolve my issue. We are seeing this kind of error because somewhere on the backend python script that we wrote has some error like ValueError.

Once I opened the cloudwatch logs, it was puzzling and I couldnt resolve the issue. But I searched for the term 'Error' in the search bar. On search, I was able to list the error and I had ValueError.

The reason for ValueError is, I tried a tutorial. The example predict_fn returned a array of two elements, [prediction, pred_prob]. The second element pred_prob had 2 elements and the first element 'prediction' had only one element. This caused dimension issues for me.

Hope this helps!

[seanpmorgan]

Closing as this was resolved. Thanks!