Prev: [PATCH -next] tile: set ARCH_KMALLOC_MINALIGN
Next: [PATCH] Revert "writeback: limit write_cache_pages integrity scanning to current EOF"
From: Grant Likely on 6 Jul 2010 04:20
On Tue, Jul 6, 2010 at 10:07 PM, Andres Salomon <dilinger(a)queued.net> wrote:
> On Wed, 30 Jun 2010 15:52:51 -0600
> Grant Likely <grant.likely(a)secretlab.ca> wrote:
> �- For the pdt, calling into the prom twice for each property/node,
> � getting the size of everything during the first tree traversal,
> � allocating memory, and then creating the tree during the second
> � traversal. This is slow and I don't see much point to it.
>
> �- For the pdt, calling into the prom once for each property/node to
> � create a fdt, and then unflattening it. �This is better than the
> � previous option, but I don't think the prom->fdt code will be very
> � nice.

No, don't do this. It will be ugly.

> �- Changing the fdt code to allocate memory more dynamically; rather
> � than a first run through the fdt to determine the unflattened tree
> � size, simply allocate memory chunks as we run through the tree.
> � Memory might still be backed the same way (calling
> � early_init_dt_alloc_memory_arch to grab a chunk of memory, and then
> � allocating chunks of it via unflatten_dt_alloc), but the difference
> � would be that it wouldn't necessarily be contiguous, and in how it's
> � done. �For example, early_init_dt_alloc_memory_arch might be called
> � initially to allocate a 16k chunk, and once unflatten_dt_alloc runs
> � out of memory in that chunk, another 16k chunk might be allocated.
> � This would also translate to sparc's prom_early_alloc being used to
> � allocate the chunk, and then node/property structs being
> � allocated from those chunks. �(The 16k choice
> � is completely arbitrary; people more familiar w/ the early memory
> � setup of ppc/sparc might have a better suggestion)

I'm fine with option 3, but it is more complicated and the code for
option 1 already exists. In fact, option 1 is almost of a
pre-requisite for doing option 3. It makes sense to get the
double-pass method factored out of powerpc and working before
attempting the performance enhancement of option 3.

Note however, the fdt data format parser expects a single contiguous
region. You'll want to copy it into a single region before parsing
it.

That Being Said....

I just finished talking with Ben about the parser code. There is no
immediate imperative to generalizing the powerpc OF->fdt->unflatten
code (unless David decides it makes sense for sparc too), and the
possibility of an ARM machine using it is theoretical enough that I'm
just not going to waste cycles thinking about it. If the situation
changes in a couple of year or two, then it can be refactored.

That clears the way for your original pdt patch. It will need to be
respun for Stephen's CONFIG_OF changes, and I have some other
comments, but I'll reply to the patches directly for those.

Cheers,
g.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Grant Likely on 6 Jul 2010 05:30
On Mon, Jun 28, 2010 at 8:00 PM, Andres Salomon <dilinger(a)queued.net> wrote:
>
> Stick code into drivers/of/pdt.c (Prom Device Tree) that other
> architectures with OpenFirmware resident in memory can make use of.
>
> Signed-off-by: Andres Salomon <dilinger(a)queued.net>

Some more comments below...

BTW, for the changes, you might what to split this patch into two
pieces. One to verbatim copy the code from prom_common.c, and a
second to adapt it for non-sparc. That will make the changes obvious
to future readers. It will also make review easier.

> ---
> �arch/sparc/Kconfig � � � � � � �| � �1 +
> �arch/sparc/include/asm/prom.h � | � 15 +++-
> �arch/sparc/kernel/prom.h � � � �| � 14 ---
> �arch/sparc/kernel/prom_common.c | �173 +------------------------------
> �drivers/of/Kconfig � � � � � � �| � �4 +
> �drivers/of/Makefile � � � � � � | � �1 +
> �drivers/of/pdt.c � � � � � � � �| �225 +++++++++++++++++++++++++++++++++++++++
> �include/linux/of_pdt.h � � � � �| � 37 +++++++
> �8 files changed, 282 insertions(+), 188 deletions(-)
> �create mode 100644 drivers/of/pdt.c
> �create mode 100644 include/linux/of_pdt.h
>
> diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
> index 6f1470b..b4cb63b 100644
> --- a/arch/sparc/Kconfig
> +++ b/arch/sparc/Kconfig
> @@ -150,6 +150,7 @@ config ARCH_NO_VIRT_TO_BUS
>
> �config OF
> � � � �def_bool y
> + � � � select OF_PROMTREE
>
> �config ARCH_SUPPORTS_DEBUG_PAGEALLOC
> � � � �def_bool y if SPARC64
> diff --git a/arch/sparc/include/asm/prom.h b/arch/sparc/include/asm/prom.h
> index f845828..0834c2a 100644
> --- a/arch/sparc/include/asm/prom.h
> +++ b/arch/sparc/include/asm/prom.h
> @@ -18,6 +18,7 @@
> �* 2 of the License, or (at your option) any later version.
> �*/
> �#include <linux/types.h>
> +#include <linux/of_pdt.h>
> �#include <linux/proc_fs.h>
> �#include <linux/mutex.h>
> �#include <asm/atomic.h>
> @@ -65,8 +66,18 @@ extern struct device_node *of_console_device;
> �extern char *of_console_path;
> �extern char *of_console_options;
>
> -extern void (*prom_build_more)(struct device_node *dp, struct device_node ***nextp);
> -extern char *build_full_name(struct device_node *dp);
> +/* stuff used by of/pdt */
> +extern void * prom_early_alloc(unsigned long size);
> +extern void irq_trans_init(struct device_node *dp);
> +extern char *build_path_component(struct device_node *dp);
> +
> +extern char *prom_firstprop(int node, char *buffer);
> +extern char *prom_nextprop(int node, const char *oprop, char *buffer);
> +extern int prom_getproplen(int node, const char *prop);
> +extern int prom_getproperty(int node, const char *prop,
> + � � � � � � � � � � � � � char *buffer, int bufsize);
> +extern int prom_getchild(int node);
> +extern int prom_getsibling(int node);

These become the API required by of/pdt. They should be defined in a
arch-independent header file. Something like include/linux/of_pdt.h

Right now only OLPC will be using this, so static function definitions
are just fine. However, if there is ever more than one method for
talking to OFW, then these hooks will need to be converted into an ops
structure so the right one can be passed in at runtime.

> diff --git a/drivers/of/pdt.c b/drivers/of/pdt.c
> new file mode 100644
> index 0000000..b24167f
> --- /dev/null
> +++ b/drivers/of/pdt.c
> @@ -0,0 +1,225 @@
> +/*
> + * Paul Mackerras � � �August 1996.
> + * Copyright (C) 1996-2005 Paul Mackerras.
> + *
> + * �Adapted for 64bit PowerPC by Dave Engebretsen and Peter Bergner.
> + * � �{engebret|bergner}@us.ibm.com
> + *
> + * �Adapted for sparc by David S. Miller davem(a)davemloft.net
> + * �Adapted for multiple architectures by Andres Salomon <dilinger(a)queued.net>
> + *
> + * � � �This program is free software; you can redistribute it and/or
> + * � � �modify it under the terms of the GNU General Public License
> + * � � �as published by the Free Software Foundation; either version
> + * � � �2 of the License, or (at your option) any later version.
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/errno.h>
> +#include <linux/mutex.h>
> +#include <linux/slab.h>
> +#include <linux/of.h>
> +#include <linux/of_pdt.h>
> +#include <asm/prom.h>
> +
> +/* TODO: mark this stuff as __initdata */
> +void (*prom_build_more)(struct device_node *dp, struct device_node ***nextp);

So do it. :-)

> +
> +#if defined(CONFIG_SPARC)
> +static unsigned int prom_unique_id;
> +
> +#define inc_unique_id(p) do { \
> + � � � (p)->unique_id = prom_unique_id++; \
> +} while (0)
> +
> +static inline const char *fetch_node_name(struct device_node *dp)
> +{
> + � � � return dp->path_component_name;
> +}
> +
> +#else
> +
> +#define inc_unique_id(p) � � � do { } while(0)

Use an empty static inline instead.

> +
> +static inline const char *fetch_node_name(struct device_node *dp)
> +{
> + � � � return dp->name;
> +}
> +
> +static inline void irq_trans_init(struct device_node *dp)
> +{
> + � � � /* unused on non-SPARC architectures */
> +}
> +
> +#endif /* !CONFIG_SPARC */
> +
> +static struct property * __init build_one_prop(phandle node, char *prev,
> + � � � � � � � � � � � � � � � � � � � � � � �char *special_name,
> + � � � � � � � � � � � � � � � � � � � � � � �void *special_val,
> + � � � � � � � � � � � � � � � � � � � � � � �int special_len)

I'd like to see all the static symbols in this file prefixed with
something unique to avoid collisions with the global namespace.
of_pdt_* would be good. I'm doing this with the other files in
drivers/of whenever I can too.

> +{
> + � � � static struct property *tmp = NULL;
> + � � � struct property *p;
> + � � � const char *name;
> +
> + � � � if (tmp) {
> + � � � � � � � p = tmp;
> + � � � � � � � memset(p, 0, sizeof(*p) + 32);
> + � � � � � � � tmp = NULL;
> + � � � } else {
> + � � � � � � � p = prom_early_alloc(sizeof(struct property) + 32);
> + � � � � � � � inc_unique_id(p);
> + � � � }
> +
> + � � � p->name = (char *) (p + 1);
> + � � � if (special_name) {
> + � � � � � � � strcpy(p->name, special_name);
> + � � � � � � � p->length = special_len;
> + � � � � � � � p->value = prom_early_alloc(special_len);
> + � � � � � � � memcpy(p->value, special_val, special_len);
> + � � � } else {
> + � � � � � � � if (prev == NULL) {
> + � � � � � � � � � � � name = prom_firstprop(node, p->name);
> + � � � � � � � } else {
> + � � � � � � � � � � � name = prom_nextprop(node, prev, p->name);
> + � � � � � � � }
> +
> + � � � � � � � if (!name || strlen(name) == 0) {
> + � � � � � � � � � � � tmp = p;
> + � � � � � � � � � � � return NULL;
> + � � � � � � � }
> +#ifdef CONFIG_SPARC32
> + � � � � � � � strcpy(p->name, name);
> +#endif
> + � � � � � � � p->length = prom_getproplen(node, p->name);
> + � � � � � � � if (p->length <= 0) {
> + � � � � � � � � � � � p->length = 0;
> + � � � � � � � } else {
> + � � � � � � � � � � � int len;
> +
> + � � � � � � � � � � � p->value = prom_early_alloc(p->length + 1);
> + � � � � � � � � � � � len = prom_getproperty(node, p->name, p->value,
> + � � � � � � � � � � � � � � � � � � � � � � �p->length);
> + � � � � � � � � � � � if (len <= 0)
> + � � � � � � � � � � � � � � � p->length = 0;
> + � � � � � � � � � � � ((unsigned char *)p->value)[p->length] = '\0';
> + � � � � � � � }
> + � � � }
> + � � � return p;
> +}
> +
> +static struct property * __init build_prop_list(phandle node)
> +{
> + � � � struct property *head, *tail;
> +
> + � � � head = tail = build_one_prop(node, NULL,
> + � � � � � � � � � � � � � � � � � �".node", &node, sizeof(node));
> +
> + � � � tail->next = build_one_prop(node, NULL, NULL, NULL, 0);
> + � � � tail = tail->next;
> + � � � while(tail) {
> + � � � � � � � tail->next = build_one_prop(node, tail->name,
> + � � � � � � � � � � � � � � � � � � � � � NULL, NULL, 0);
> + � � � � � � � tail = tail->next;
> + � � � }
> +
> + � � � return head;
> +}
> +
> +static char * __init get_one_property(phandle node, const char *name)
> +{
> + � � � char *buf = "<NULL>";
> + � � � int len;
> +
> + � � � len = prom_getproplen(node, name);
> + � � � if (len > 0) {
> + � � � � � � � buf = prom_early_alloc(len);
> + � � � � � � � len = prom_getproperty(node, name, buf, len);
> + � � � }
> +
> + � � � return buf;
> +}
> +
> +struct device_node * __init prom_create_node(phandle node,
> + � � � � � � � � � � � � � � � � � � � � � �struct device_node *parent)
> +{
> + � � � struct device_node *dp;
> +
> + � � � if (!node)
> + � � � � � � � return NULL;
> +
> + � � � dp = prom_early_alloc(sizeof(*dp));
> + � � � inc_unique_id(dp);
> + � � � dp->parent = parent;
> +
> + � � � kref_init(&dp->kref);
> +
> + � � � dp->name = get_one_property(node, "name");
> + � � � dp->type = get_one_property(node, "device_type");
> + � � � dp->phandle = node;
> +
> + � � � dp->properties = build_prop_list(node);
> +
> + � � � irq_trans_init(dp);
> +
> + � � � return dp;
> +}
> +
> +static char * __init build_full_name(struct device_node *dp)
> +{
> + � � � int len, ourlen, plen;
> + � � � char *n;
> +
> + � � � plen = strlen(dp->parent->full_name);
> + � � � ourlen = strlen(fetch_node_name(dp));
> + � � � len = ourlen + plen + 2;
> +
> + � � � n = prom_early_alloc(len);
> + � � � strcpy(n, dp->parent->full_name);
> + � � � if (!is_root_node(dp->parent)) {
> + � � � � � � � strcpy(n + plen, "/");
> + � � � � � � � plen++;
> + � � � }
> + � � � strcpy(n + plen, fetch_node_name(dp));
> +
> + � � � return n;
> +}
> +
> +struct device_node * __init prom_build_tree(struct device_node *parent,
> + � � � � � � � � � � � � � � � � � � � � � phandle node,
> + � � � � � � � � � � � � � � � � � � � � � struct device_node ***nextp)
> +{
> + � � � struct device_node *ret = NULL, *prev_sibling = NULL;
> + � � � struct device_node *dp;
> +
> + � � � while (1) {
> + � � � � � � � dp = prom_create_node(node, parent);
> + � � � � � � � if (!dp)
> + � � � � � � � � � � � break;
> +
> + � � � � � � � if (prev_sibling)
> + � � � � � � � � � � � prev_sibling->sibling = dp;
> +
> + � � � � � � � if (!ret)
> + � � � � � � � � � � � ret = dp;
> + � � � � � � � prev_sibling = dp;
> +
> + � � � � � � � *(*nextp) = dp;
> + � � � � � � � *nextp = &dp->allnext;
> +
> +#if defined(CONFIG_SPARC)
> + � � � � � � � dp->path_component_name = build_path_component(dp);
> +#endif
> + � � � � � � � dp->full_name = build_full_name(dp);
> +
> + � � � � � � � dp->child = prom_build_tree(dp, prom_getchild(node), nextp);
> +
> + � � � � � � � if (prom_build_more)
> + � � � � � � � � � � � prom_build_more(dp, nextp);
> +
> + � � � � � � � node = prom_getsibling(node);
> + � � � }
> +
> + � � � return ret;
> +}

prom_build_devicetree() should also be factored out.

> diff --git a/include/linux/of_pdt.h b/include/linux/of_pdt.h
> new file mode 100644
> index 0000000..f62616e
> --- /dev/null
> +++ b/include/linux/of_pdt.h
> @@ -0,0 +1,37 @@
> +#include <linux/of.h> �/* linux/of.h gets to determine #include ordering */

Do you really need this #include in this way? Can it be moved inside
the #ifndef OF_PDT block below?

> +/*
> + * Definitions for building a device tree by calling into the
> + * Open Firmware PROM.
> + *
> + * Copyright (C) 1996-2005 Paul Mackerras.
> + * Copyright (C) 2010 �Andres Salomon <dilinger(a)queued.net>
> + *
> + * Updates for PPC64 by Peter Bergner & David Engebretsen, IBM Corp.
> + * Updates for SPARC by David S. Miller
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version
> + * 2 of the License, or (at your option) any later version.
> + */
> +
> +#ifndef _LINUX_OF_PDT_H
> +#define _LINUX_OF_PDT_H
> +
> +extern struct device_node *prom_create_node(phandle node,
> + � � � � � � � � � � � � � � � � � � � � � struct device_node *parent);
> +extern struct device_node *prom_build_tree(struct device_node *parent,
> + � � � � � � � � � � � � � � � � � � � � �phandle node,
> + � � � � � � � � � � � � � � � � � � � � �struct device_node ***nextp);
> +
> +extern void (*prom_build_more)(struct device_node *dp, struct device_node ***nextp);
> +
> +static inline int is_root_node(const struct device_node *dp)
> +{
> + � � � if (!dp)
> + � � � � � � � return 0;
> +
> + � � � return (dp->parent == NULL);
> +}

This doesn't look like a pdt specific function, it could go in
include/linux/of.h.

Another general comment, I'm still not thrilled with this code having
its own independent method for building the tree, but I doubt the
existing add/remove nodes and properties code is usable early enough
to be suitable for sparc. How early do you extract the device tree on
OLPC? How are you going to use the data?

Anyway, enough comments for tonight. I'll may have more on then next
round. Time to go to bed.

g.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
From: Grant Likely on 6 Jul 2010 18:10
On Tue, Jul 6, 2010 at 3:54 PM, Andres Salomon <dilinger(a)queued.net> wrote:
> On Tue, 6 Jul 2010 03:21:21 -0600
> Grant Likely <grant.likely(a)secretlab.ca> wrote:
>
>> On Mon, Jun 28, 2010 at 8:00 PM, Andres Salomon <dilinger(a)queued.net>
>> wrote:
>> >
>> > Stick code into drivers/of/pdt.c (Prom Device Tree) that other
>> > architectures with OpenFirmware resident in memory can make use of.
>> >
>> > Signed-off-by: Andres Salomon <dilinger(a)queued.net>
>>
>> Some more comments below...
>>
>
> Thanks for the review!
>
>
>
>> > �arch/sparc/Kconfig � � � � � � �| � �1 +
>> > �arch/sparc/include/asm/prom.h � | � 15 +++-
>> > �arch/sparc/kernel/prom.h � � � �| � 14 ---
>> > �arch/sparc/kernel/prom_common.c | �173
>> > +------------------------------ drivers/of/Kconfig � � � � � � �|
>> > �4 + drivers/of/Makefile � � � � � � | � �1 +
>> > �drivers/of/pdt.c � � � � � � � �| �225
>> > +++++++++++++++++++++++++++++++++++++++ include/linux/of_pdt.h
>> > � � �| � 37 +++++++ 8 files changed, 282 insertions(+), 188
>> > deletions(-) create mode 100644 drivers/of/pdt.c
>> > �create mode 100644 include/linux/of_pdt.h
>> >
>> > diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
>> > index 6f1470b..b4cb63b 100644
>> > --- a/arch/sparc/Kconfig
>> > +++ b/arch/sparc/Kconfig
>> > @@ -150,6 +150,7 @@ config ARCH_NO_VIRT_TO_BUS
>> >
>> > �config OF
>> > � � � �def_bool y
>> > + � � � select OF_PROMTREE
>> >
>> > �config ARCH_SUPPORTS_DEBUG_PAGEALLOC
>> > � � � �def_bool y if SPARC64
>> > diff --git a/arch/sparc/include/asm/prom.h
>> > b/arch/sparc/include/asm/prom.h index f845828..0834c2a 100644
>> > --- a/arch/sparc/include/asm/prom.h
>> > +++ b/arch/sparc/include/asm/prom.h
>> > @@ -18,6 +18,7 @@
>> > �* 2 of the License, or (at your option) any later version.
>> > �*/
>> > �#include <linux/types.h>
>> > +#include <linux/of_pdt.h>
>> > �#include <linux/proc_fs.h>
>> > �#include <linux/mutex.h>
>> > �#include <asm/atomic.h>
>> > @@ -65,8 +66,18 @@ extern struct device_node *of_console_device;
>> > �extern char *of_console_path;
>> > �extern char *of_console_options;
>> >
>> > -extern void (*prom_build_more)(struct device_node *dp, struct
>> > device_node ***nextp); -extern char *build_full_name(struct
>> > device_node *dp); +/* stuff used by of/pdt */
>> > +extern void * prom_early_alloc(unsigned long size);
>> > +extern void irq_trans_init(struct device_node *dp);
>> > +extern char *build_path_component(struct device_node *dp);
>> > +
>> > +extern char *prom_firstprop(int node, char *buffer);
>> > +extern char *prom_nextprop(int node, const char *oprop, char
>> > *buffer); +extern int prom_getproplen(int node, const char *prop);
>> > +extern int prom_getproperty(int node, const char *prop,
>> > + � � � � � � � � � � � � � char *buffer, int bufsize);
>> > +extern int prom_getchild(int node);
>> > +extern int prom_getsibling(int node);
>>
>> These become the API required by of/pdt. �They should be defined in a
>> arch-independent header file. �Something like include/linux/of_pdt.h
>>
>> Right now only OLPC will be using this, so static function definitions
>> are just fine. �However, if there is ever more than one method for
>> talking to OFW, then these hooks will need to be converted into an ops
>> structure so the right one can be passed in at runtime.
>>
>
> Note that sparc and OLPC actually use slightly different function
> signatures; OLPC uses phandles for nodes, while sparc uses ints. �Not a
> huge difference, but enough that I didn't want to mess w/ a generic
> version of it early in the process.

phandle is simply defined as a u32. It probably wouldn't be difficult
to change the sparc code to use phandle; redefining the type for sparc
if need be.

> �I agree that op structs would be
> nicer, and will probably move towards that.
>
> [...]
>
>> > diff --git a/include/linux/of_pdt.h b/include/linux/of_pdt.h
>> > new file mode 100644
>> > index 0000000..f62616e
>> > --- /dev/null
>> > +++ b/include/linux/of_pdt.h
>> > @@ -0,0 +1,37 @@
>> > +#include <linux/of.h> �/* linux/of.h gets to determine #include
>> > ordering */
>>
>> Do you really need this #include in this way? �Can it be moved inside
>> the #ifndef OF_PDT block below?
>>
>
> Not sure, will try out different variants and see what breaks.
>
> [...]
>
>> Another general comment, I'm still not thrilled with this code having
>> its own independent method for building the tree, but I doubt the
>> existing add/remove nodes and properties code is usable early enough
>> to be suitable for sparc. �How early do you extract the device tree on
>> OLPC? �How are you going to use the data?
>
> Not that early; very early code fetches information necessary to call
> into the PROM, and ensures that the kernel doesn't clobber OFW's
> memory. � After that, we can build the dt at any point during init.
> The data is to be exported via proc for userspace to use in
> determining hardware (and firmware) info.

Okay, so it is only the userspace interface that you're interested in,
correct? You have no needs/plans (as of yet) to register devices out
of the device tree?

g.

--
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo(a)vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
First  |  Prev  | 
Pages: 1 2
Prev: [PATCH -next] tile: set ARCH_KMALLOC_MINALIGN
Next: [PATCH] Revert "writeback: limit write_cache_pages integrity scanning to current EOF"