Archives of Design Research

The parametric setting that governs the morphological characteristics of a cell interface is a key factor that enables immersive user interaction. From a usability perspective, it is necessary that the interface maintains a certain level of sensitivity and responsiveness so that it can adapt in real-time to input from a user’s sensory actions. Accordingly, to determine an optimal degree of responsiveness, Table 1 presents a simulation focusing on how the spikes of a cell interface extend in response to the upward movement of a user’s hand.

Input Parameters	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5	Stage 6	Stage 7	Stage 8
<Parameter 1> Distance value between index finger’s point and the center point of a 2d surface on the ground(height of a hand)	15.572355	15.6838	15.624299	15.766315	15.52455	15.763275	15.725905	15.568403
<Parameter 2> ‘Weighting’ option of ‘Load’ component	0.055	0.1103	0.1986	0.2386	0.3124	0.339	0.4016	0.4316
<Parameter 3> ‘Strength’ option of ‘Anchor’ component	90.76	90.76	90.76	90.76	90.76	90.76	90.76	90.76
<Parameter 4> ‘Distance’ option of ‘Weaverbird’s Stellate/Cumulation’ component	0.798	2.834	6.149	10.049	18.583	29.633	46.798	59.008
<Parameter 5> ‘Length’ option of ‘Trimesh’ component	3	3	3	3	3	3	3	3

Within this simulation, five Grasshopper components serve as the main input parameters: the distance between the index finger’s skeletal point and the center of a 2D ground plane (i.e., the user’s hand height); the “Weighting” option within the “Load” component; the “Strength” option in the “Anchor” component; the “Distance” option of the “Weaverbird’s Stellate/Cumulation” component; and the “Length” option of a “Trimesh” component. These are all linked to number sliders for adjustable numerical inputs. The value representing the hand’s height is captured via skeletal data (index finger to 2D surface center on the XY plane), and its numerical output from the “Distance” component is relayed to the “Sync Text Tag” component of a Fologram plugin for real-time display.

Figure 4 Cell spike elongation in response to a hand-lifting gesture. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

To monitor changes in the spikes’ elongation, several conditions are preset. First, parameter 3 remains at 90.76 to anchor the mesh structure to the surface. Second, the hand’s vertical distance (parameter 1) stays between 15.00 and 16.00, allowing for clear observation of morphological alterations driven by parameters 2 and 4. Since parameters 2 and 4 critically influence spike elongation, they are incrementally increased at each stage, while parameter 3 remains constant. Under these circumstances, the mesh exhibits progressive volumetric expansion and remains anchored to the 2D ground plane. In addition, to sustain the properly-inflated turgidity of a cell interface and inhibit unexpected morphological deformation by mesh explosion, parameter 1’s distance value variation is maintained with slight fluctuation between 15.52455 and 15.766315.

Categorizing non-linear hand gestures provides a guiding framework for developing gesture recognition. During this stage, diverse hand movements detected by the Leap Motion controller are organized as typological patterns (Figure 5). Integrated with plant cell-inspired morphological attributes (Figure 2), the interface responds to predetermined gestures—like clenching and opening the fist (and vice versa), hovering, and moving the hand up and down—by enacting specific morphological reactions.

Figure 5 Typological categorization of hand gestures for interacting with a cell interface using the Leap Motion controller. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Establishing a gesture-oriented simulation, the morphological dynamics of a cell interface are defined by a hormone-influenced growth regulation framework found in plant biology. Different hormone signals and their spatial distribution cause varying growth results (Zhu et al., 2020). Additionally, the presence and concentration of plant hormones like Auxin and Abscisic Acid (ABA) modulate the growth and proliferation of cells (Yue et al., 2021). Drawing on these principles, 2D cell interface prototypes are generated and assessed under various conditions. For modeling a simulation, a “Kangaroo Physics” plugin, and “SphereCollide,” “Voronoi,” “CurvePointCollide” components facilitate modeling both cell proliferation and growth inhibition (see lower right of Figure 7).

Figure 6 displays how cell interfaces proliferate under different scenarios. Image 1 shows an increase in cell density (from 30 to 250) by increasing the numerical value connected to a “Count” option of a “Populate Geometry” component to simulate cell division. In Image 2, a red cell wall is introduced to hinder further proliferation by linking a curve to the “CurvePointCollide” component, reducing expansion. In Image 3, multiple cell growth emission areas produce alternative proliferation outcomes.

Figure 6 Sequential demonstration of cell proliferation models: (1) default proliferation, (2) a cell wall introduced to limit growth, (3) multiple cell wall structures enabling selective growth areas. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Figure 7 Grasshopper scripting for cell proliferation. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

The mechanism discussed in 3.1.3, focusing on cell proliferation and an inhibiting cell wall, transitions to a 3D framework (Figure 8). In response to a user’s vertical hand movement, the region that fostered cell proliferation in 2D is transformed into a green, inflated geometry that expands toward the raising hand. In contrast, the region acting as an inhibitor becomes a blue, non-inflated geometry that demonstrates minimal response to the same vertical movement. By adjusting how the surface inflation mechanism reacts, this coded inhibitory region manages the interface’s morphological development. The outcomes become progressively distinct across the five stages as the hand-lifting distance increases (Table 2).

Figure 8 Conceptual diagram showing how 2D proliferation models transition into 3D cell proliferation models with regulatory systems. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Input Parameters	Stage 1		Stage 2		Stage 3		Stage 4		Stage 5
	Top Area	Bottom Area	Top Area	Bottom Area	Top Area	Bottom Area	Top Area	Bottom Area	Top Area	Bottom Area
<Parameter 1> Distance value between index finger’s point and the center point of a 2d surface on the ground(height of a hand)	11.754076		16.975323		23.490336		33.70654		43.910953
<Parameter 2> ‘Weighting’ option of ‘Load’ component	0.3146	0.1524	0.3146	0.1524	0.3146	0.1524	0.3146	0.1524	0.3146	0.1524
<Parameter 3> ‘Strength’ option of ‘Anchor’ component	98.4		98.4		98.4		98.4		98.4
<Parameter 4> ‘Distance’ option of ‘Weaverbird’s Stellate/Cumulation’ component	-25.2	10.955	-25.	10.955	-25.2	10.955	-25.2	10.955	-25.2	10.955
<Parameter 5> ‘Length’ option of ‘Trimesh’ component	3		3		3		3		3
Result Value	Stage 1		Stage 2		Stage 3		Stage 4		Stage 5
Height value of an inflated mesh structure (Distance between inflated mesh structure’s mesh point and its projected point on the XY plane)	11.830459	11.830459	15.48973	10.071187	18.954637	11.482741	22.594235	12.933366	26.266357	14.328323

Similar to 3.1.1, the distance between the hand and the cell interface is calculated using the index finger’s skeletal data and the center of a 2D circle comprising two mesh types. Unlike 3.1.1’s regulated range of a height of a hand, the height consistently increases with each stage, leading parameter 1 to climb while parameters 2 through 5 remain unchanged. Notably, parameter 4 is configured to implement opposite inflation directions for top and bottom sections, assigning negative values to the top region and positive to the bottom. Meanwhile, parameters 2 and 3 remain static to maintain mesh stability and preserve its attachment to the platform.

To gauge the distance between the user’s hand and the interface, mesh points on the inflated structure are initially labeled with Grasshopper’s “Point List” (Figure 9). Subsequently, these points (from both top and bottom regions) are projected onto the XY plane. The distance from each original mesh point to its ground projection is then recorded as the inflated mesh’s height (Figure 10).

Figure 9 Mesh point tagging on inflated structures for identification. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Figure 10 Determining the inflated mesh’s height by measuring the space between a mesh point and its ground projection (in green lines to denote vertical distance). Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

By examining the structural responsiveness of these meshes, height results listed in Table 2 are charted as line graphs in Chart 1. The comparison of gradients between the top and bottom areas shows that the graph for the top area has a slightly steeper slope, signifying that the top area’s mesh inflation is more sensitive to the incremental vertical movement of the hand compared to the bottom area. Moreover, as detailed in 3.1.4, the differential settings for the top and bottom areas (with negative and positive values in parameter 4) are intentionally configured to showcase how areas promoting cell growth and those inhibiting it can respond differently to the same hand motion.

Chart 1 Graph comparing the structural responsiveness of top and bottom meshes to increasing hand height. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Building on the hormone-based growth regulatory system introduced in 3.1.3, this phase applies the same principle to guide the morphological behavior and proliferation of cell interfaces. By leveraging the interplay between cell growth accelerators and inhibitors, this simulation focuses on two core concepts: (a) Cell interfaces that proliferate through a spatial cell-growth-accelerating region; (b) Users who function as inhibitors, regulating and limiting cell growth within a given space. In this simulation, a user essentially acts as a cell growth inhibitor while the cell interface freely disperses throughout the environment (Figure 11). The user’s occupied area—projected onto a 2D plane—blocks the expansion of cell centroids and constrains their directional vectors. This setup mirrors the cell wall mechanism from 3.1.3, where the user’s spatial footprint replaces the cell wall, governing cell proliferation and affecting the fluid-like motion of the interface in real time.

Figure 11 Simulations illustrating how a user can impede cell proliferation or influence collective cell direction by forming a 2D area (shown as a red “blob” around the user’s skeleton) on the ground where the cell interface grows. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

To create this system, the same algorithmic structure used in 3.1.3 is reused, augmented by the Metaball component for visually fluid cell interface growth. In the Grasshopper script, the cell interface’s centroids feed into the “points” input of the “Metaball” component, forming line-based fluid structures that interact with the user’s footprint. Notably, the cell interface scatters when the user walks through it, showing a fluid reaction triggered by the Metaball function.

Regarding user data, Figure 11 shows how skeletal points captured by Kinect V2 are projected onto the ground plane (XY Plane). These projected circles merge into a single blob-like shape using the “Region Union” component. Finally, linking this unified 2D geometry to the “CurvePointCollide” component designates it as a collision object for the cells. As a result, this 2D surface derived from the user’s skeleton serves as a cell growth inhibitor, curbing the interface’s expansion or steering the directional paths of proliferating cells (Figure 12).

Figure 12 Grasshopper script generating a user’s occupied area based on skeletal data. The red surface enveloping the skeleton in Figure 11 represents how a user’s space influences the collective movement of cells. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Using the same developmental logic, the simulation now scales up to incorporate multiple users and real-world architectural environments. Within the script, several new input variables are introduced: (a) Detection of multiple individuals; (b) Recognition of actual spatial conditions. Kinect V2 tracks each participant’s body motions, converting them into skeletons on the Rhino3D platform through the “Kinect V2 Skeleton Tracker” in Firefly. As in 3.2.1, these users act as cell growth inhibitors, limiting how far the cell interface can expand. Specifically, the 2D projection of users’ occupied areas interrupts and segments the Metaball blob, influencing the overall direction of the interface’s cell centroids (Figure 13).

Figure 13 Multi-user simulations of a cell interface regulated by a hormone-based growth system. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

In this multi-object simulation, architectural settings also serve as an inhibitory factor, comparable to the user-generated zones, thereby affecting the interface’s proliferation and shape. Additionally, the inhibiting area responding to the environment may adapt in a generative manner, depending on environmental conditions. To map the actual surroundings for Mixed Reality simulations, Aruco Markers are placed on the ground corresponding to the layout of real-world architecture (Figure 14).

Figure 14 Aruco Marker pads used for mapping physical environments. To enhance stability, 3D-printed pads (15 mm thick) are attached to the markers before laying them out on the ground. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

A small-scale simulation (Figure 15) demonstrates how environmental volumes are detected through Aruco Markers and subsequently influence the cell interface’s proliferation. By placing Aruco Markers on the ground near physical objects, devices running the Fologram app (smartphone, iPad, or HoloLens2 HMD) scan the markers’ XYZ positions and relay them to Grasshopper in real time. These points become control vertices for generating curves, which then form surfaces (the blue areas in Figure 15). Together with user-occupied zones, these environmental surfaces act as inhibitors, limiting the spread of the cell interface’s centroids as they proliferate.

Figure 15 Speculative simulation of how Aruco Markers map the environment, restricting the cell interface’s growth. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

To enhance the visual portrayal of proliferation, the interface employs a 3D Metaball system based on line segments (Figure 16). In Grasshopper, the “Metaball(t) Custom” component is used alongside an array of reference XY planes spaced perpendicularly on the platform, creating a layered Metaball structure. Assigning varying “charging” values to each cell centroid with the “Gene Pool” component controls the density of these stacked Metaball lines.

Figure 16 Speculative simulation showing how a 3D Metaball fluid system animates the proliferation of a cell interface constrained by user interactions and environmental factors. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

As explained in 3.2.2, a set of Aruco Marker pads (approximately 16 to 24, with markers affixed to 3D-printed pads) is prepared and placed near physical objects and architectural features such as beams, columns, chairs, and tables (Images 1 and 2 of Figure 17). Once the markers are arranged, a user with an iPad running the Fologram app, or wearing a HoloLens2 HMD with the same app, begins scanning each marker at close range. As the markers are scanned, their positions are converted into XYZ coordinates, which appear as points in Rhino3D in real time (see the red points near the generated red surface in Image 4). Meanwhile, the unique marker labels are compiled in Grasshopper’s recording panel (the yellow panel on the right of Image 5). By entering integer values in the “Degree” and “Periodic” options for the “Nurbs Curve” component, the newly created points yield a nurbs curve, which is then unified into a single closed curve. This curve is subsequently turned into a surface (the red surface in Image 4). Consequently, anyone wearing the HoloLens2 HMD sees this surface blended into the real environment through mixed reality (Image 3 of Figure 17).

Figure 17 Process illustrating how arrayed Aruco Markers detect the physical environment and transform it into a red surface in Mixed Reality. This generated surface, which represents the physical environment, influences both the proliferation of the cell interface and the multidirectional movement of its centroids. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

In Image 3 of Figure 17, the nurbs curve from the Aruco Markers is linked to the “Curve” option of the “CurvePointCollide” component. As shown in Figures 15 and 16, the resulting red surface signifies the physical environment, functioning as a growth inhibitor that constrains the cell interface’s local proliferation—akin to the user’s blob area that shapes morphological changes in the interface. In conjunction with the HoloLens2 HMD’s mixed reality functionality, the interface’s proliferation and reactions to the recognized red surface (derived from real environmental elements) are rendered in real time.

Relying on this method of detecting the physical environment and converting it into a 2D graphic surface in MR, simulations of a cell growth regulatory system-driven interface are carried out (Figure 18). Viewing the simulation from a third-person perspective in the Rhino3D modeling space (Image 1 of Figure 18), the user wearing a HoloLens2 HMD interacts with the holographic cell interface layered over the physical setting.

Figure 18 Simulations of a growth-regulated cell interface that responds to user body movements and physical surroundings identified by Aruco Markers. (1) Real-time simulation in Rhino3D’s modeling space while placing a full-scale interface in the actual environment, (2) A metaball fluid structure reacting to the user’s movement, (3) Fluid interaction of the metaball-based interface as it encounters the environment’s area (shown as a faint red surface on the left) defined by Aruco Markers, (4) Line-based metaball structure rendered in Mixed Reality, displaying fluid behavior. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

Within the mixed reality environment, a metaball fluid structure integrated into the cell interface’s proliferation model engages with user gestures—like walking or clapping—by tracking skeletal frames through Kinect V2. Moreover, the metaball structure’s growth and motion are deflected by the red surface formed by the spatial arrangement of Aruco Markers. Additionally, to react to a user’s movements on the floor, the user’s 2D footprint (calculated from Kinect V2’s skeleton data) is recorded in Grasshopper as a prerequisite. As a result, this 2D footprint functions as a growth inhibitor, affecting both the interface’s proliferation and the metaball structure’s fluid reactions in an interactive manner (Image 2 of Figure 18).

Subsequently, the simulation of the growth-regulated cell interface expands into multi-user scenarios under the same developmental and environmental criteria (Figure 19). Anchored by the generated surface (green area on the left side of Image 1 in Figure 19) created with Aruco Markers around the chosen space, a metaball-based cell interface proliferates. As it does, the fluid-like morphology of the metaball structure changes upon contacting user-defined or environment-defined boundaries.

Figure 19 Multi-user simulations of a growth-regulated cell interface. (1) Users’ skeletal frames as they engage with the interface through body gestures and hand movements, (2) A close-up of the line-based fluid structure as seen through a HoloLens2 HMD, (3) A user experiencing the holographic interface overlaid in Mixed Reality, (4) A holographic interface responding to hand gestures, where proliferation rate, volumetric behavior of the metaball structure, and expansion levels are tied to the proximity values detected from the user’s moving hand skeleton. Reprinted from the first author’s master thesis“CELLNET: Bio-Inspired Adaptive Cellular Interface in Mixed Reality”, submitted to Cornell University, 2023

To preserve the volume of the metaball structure even as the cell interface expands, the maximum input value for the “Radius” setting of the “Sphere Collide” component—which determines proliferation speed and fluidity—is fixed at 108.00 when receiving user input. Values above 108.00 are purposely scaled down by multiplying with a decimal factor. This way, user inputs remain under the threshold. Under these constraints, user proximity values, measured from their hands’ skeleton points, feed into the “Radius” parameter of the “Sphere Collide” component, causing the interface to expand in step with a user’s hand movements (Image 4 of Figure 19).

To explore the interface’s site-specific adaptability, additional simulations of the growth-regulated cell interface were conducted in diverse locations with varied architectural and environmental features (Table 3). Since reliable Wi-Fi was necessary to synchronize the modeling data with the iPad or a HoloLens2 HMD, most simulations were carried out in designated indoor areas. In those spaces, permanent and temporary architectural elements—like partition walls or sponge stands for exhibitions—provide environmental factors for the interface to interact with in Mixed Reality.

Conditions	Simulation 1	Simulation 2	Simulation 3	Simulation 4	Simulation 5
MR Simulation in the actual environments
<Outcome 1> Recognized area of targeted physical environment via ‘Aruco Markers’
<Outcome 2> Voronoi-based cell interface towards recognized environmental area
<Outcome 3> 2D Metaball structure-based cell interface towards recognized environmental area
<Outcome 4> 3D Metaball structure-based cell interface towards recognized environmental area
Venue for the simulation	Dome area of Milstein Hall, Cornell AAP	Dome area of Milstein Hall, Cornell AAP	Dance floor area of Milstein Hall, Cornell AAP	2rd floor of Sibley Hall, Cornell AAP	3rd floor of Sibley Hall, Cornell AAP
Venue for the simulation	Movable partition walls, Sponge pedestal stands	Diagonal pillar, long bench	H-Beam	Movable partition walls, Pillar	Movable partition walls, Movable TV stand, Chair
Total number of Aruco Marker pads used	18ea	18ea	26ea	21ea	28ea
<Parameter 1> ‘Radius’ option of ‘Sphere Collide’ component	108.00	55.88	90.32	82.82	82.82
<Parameter 2> ‘Strength’ option of ‘Sphere Collide’ component	100.0	100.0	100.0	100.0	100.0
<Parameter 3> ‘Degree’ option of ‘Nurbs Curve’ component	1.2	1.2	7.3	7.3	7.3
<Parameter 4> ‘Count’ option of ‘Populate Geometry’ component	38	38	38	38	38
<Parameter 5> ‘Strength’ option of ‘CurvePointCollide’ component	97.534	97.534	64.874	64.874	64.874

As in the earlier simulations, an array of Aruco Marker pads is placed around the targeted zone to detect that portion of the physical environment and convert it into a regulating surface. Through this process, the Aruco Markers’ XYZ coordinates are registered in Rhino3D and Grasshopper, where they are used to generate surfaces.

Due to each site’s layout of architectural or environmental features, the recognized area in each simulation produces varied results (Outcome 1 in Table 3). Each simulation employed different edge parameters when forming the surface from the scanned Aruco Markers’ coordinates. In Parameter 3 of Table 3, the “Degree” option for the “Nurbs Curve” component was set to 1.2 in Simulation 1 and 2, yielding more angular, non-filleted edges. Conversely, Simulation 3, 4, and 5 used a same “Degree” option of 7.3, resulting in smoother, rounded edges.

Based on these differently generated surfaces—capturing real-space elements—the interface’s proliferation model was tested. Observing how the interface responded morphologically to each environment’s recognized area, the surfaces in Simulation 1, 4, and 5 had a more pronounced effect on the interface’s shape than in other cases. Structurally, the cell centroids in Simulation 1, 4, and 5 began in a compressed state, reflecting narrower corners in the recognized real-world environment. Combined with the presets in Parameter 1 and 5, which regulate the interface’s expansion rate and velocity, this congestion influenced the collective direction of the cell centroids, eventually triggering the interface to burst into more open areas. This effect is clearly visible in Outcome 4 of Simulation 4 and 5.