Generate vectorcall code to parse arguments using Argument Clinic

@vstinner

BPO	43447
Nosy	@vstinner, @serhiy-storchaka, @corona10, @erlend-aasland, @kumaraditya303

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/corona10'
closed_at = None
created_at = <Date 2021-03-09.11:34:23.944>
labels = ['expert-C-API', '3.10']
title = 'Generate vectorcall code to parse arguments using Argument Clinic'
updated_at = <Date 2022-03-06.09:38:05.594>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-03-06.09:38:05.594>
actor = 'kumaraditya'
assignee = 'corona10'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2021-03-09.11:34:23.944>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 43447
keywords = []
message_count = 2.0
messages = ['388354', '389214']
nosy_count = 5.0
nosy_names = ['vstinner', 'serhiy.storchaka', 'corona10', 'erlendaasland', 'kumaraditya']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue43447'
versions = ['Python 3.10']

Linked PRs

I agree with we should update AC to generate vectorcall.
I am going to investigate what we can :)

Dong-hee, did you ever get to this? :) If so, I'd be interested in hearing what you found out.

Dong-hee, did you ever get to this? :) If so, I'd be interested in hearing what you found out.

Sorry, I didn't make progress on this issue. One thing I considered at that moment was supporting positional arguments only cases first. Implementing keyword arguments will be quite complex and not many use cases exist we should cover. so it is not urgent.

If you want to take a look at this issue, it will be fine.

enumrate is good example when you want to handle keyword arguments. as you can read it's too complex.

cpython/Objects/enumobject.c

Lines 101 to 150 in 1344cfa

    
           enumerate_vectorcall(PyObject *type, PyObject *const *args, 
        
                                size_t nargsf, PyObject *kwnames) 
        
           { 
        
               PyTypeObject *tp = _PyType_CAST(type); 
        
               Py_ssize_t nargs = PyVectorcall_NARGS(nargsf); 
        
               Py_ssize_t nkwargs = 0; 
        
               if (kwnames != NULL) { 
        
                   nkwargs = PyTuple_GET_SIZE(kwnames); 
        
               } 
        
               // Manually implement enumerate(iterable, start=...) 
        
               if (nargs + nkwargs == 2) { 
        
                   if (nkwargs == 1) { 
        
                       if (!check_keyword(kwnames, 0, "start")) { 
        
                           return NULL; 
        
                       } 
        
                   } else if (nkwargs == 2) { 
        
                       PyObject *kw0 = PyTuple_GET_ITEM(kwnames, 0); 
        
                       if (_PyUnicode_EqualToASCIIString(kw0, "start")) { 
        
                           if (!check_keyword(kwnames, 1, "iterable")) { 
        
                               return NULL; 
        
                           } 
        
                           return enum_new_impl(tp, args[1], args[0]); 
        
                       } 
        
                       if (!check_keyword(kwnames, 0, "iterable") || 
        
                           !check_keyword(kwnames, 1, "start")) { 
        
                           return NULL; 
        
                       } 
        
                   } 
        
                   return enum_new_impl(tp, args[0], args[1]); 
        
               } 
        
               if (nargs + nkwargs == 1) { 
        
                   if (nkwargs == 1 && !check_keyword(kwnames, 0, "iterable")) { 
        
                       return NULL; 
        
                   } 
        
                   return enum_new_impl(tp, args[0], NULL); 
        
               } 
        
               if (nargs == 0) { 
        
                   PyErr_SetString(PyExc_TypeError, 
        
                       "enumerate() missing required argument 'iterable'"); 
        
                   return NULL; 
        
               } 
        
               PyErr_Format(PyExc_TypeError, 
        
                   "enumerate() takes at most 2 arguments (%d given)", nargs + nkwargs); 
        
               return NULL; 
        
           }

Sorry, I didn't make progress on this issue. One thing I considered at that moment was supporting positional arguments only cases first. Implementing keyword arguments will be quite complex and not many use cases exist we should cover. so it is not urgent.

If you want to take a look at this issue, it will be fine.

No need to be sorry :) I don't have the bandwidth to look at it right now, but I hope to be able to pick it up this fall. Let's keep the issue open. If you find time for this, that is great, but there is no urgency :)

Taking a look at this scoped down to __new__ and __init__ which should be able to do float, bytes, str, and bytearray to simplify and optimize some cases. My plan is to try making a @vectorcall decorator in clinic which causes two parse arg codegens to output: one for the existing, a second which adds an _vectorcall variant. What I'm experimenting with is the following input:

/*[clinic input]
@vectorcall
Test.__init__
    a: object
    /
    ab: object = None
Vectorcall __init__ with args and kwargs.
[clinic start generated code]*/

@vectorcall

My plan is to try making a @vectorcall decorator in clinic which causes two parse arg codegens to output: one for the existing, a second which adds an _vectorcall variant.

That sounds great, I am looking forward to use it to speed up asyncio task creation which would benefit from this.

pythongh-87613: Add vectorcall constructor benchmarks

@vectorcall

pythongh-87613: Add @vectorcall decorator support to Argument Clinic

pythongh-87613: Migrate list.__init__ to clinic-generated vectorcall

pythongh-87613: Add vectorcall to bytearray.__init__ via Argument Clinic

pythongh-87613: Migrate float.__new__ to clinic-generated vectorcall

pythongh-87613: Add vectorcall to bytes.__new__ via Argument Clinic

pythongh-87613: Migrate str.__new__ to clinic-generated vectorcall