WhitneyLand
At a glance it looks like it could be just iterative
texture sampling.The difference is when creating each
pixel, there's no coordinate to look up, instead it's
using only a set of rules like Conway's game of life.But
the rules come from a neural network trained on the image,
so... it's kind of memorizing enough information to
effectively do the same thing as texture sampling, but
using only local information.I'm sure I'm missing
something about how it works or what makes it
interesting...
|
whilenot-dev
The automata just completely destroys the image if I draw
too much over the stabilized image with the brush. 5
horizontal swipes are enough to destroy the kitty, is that
to be expected?EDIT: video here:
https://imgur.com/a/ItZGd5X
|
> esychology
The NeuralCA both generates and maintains the pattern.
Because the NCA was not exposed to damage or erasure
during training, its regeneration capability is a
purely emergent phenomenon. However, this ability
remains somewhat brittle, particularly when the
central regions of the pattern are erased.
|
> > mackenney
I would love to see two seeds competing for space
in the grid
|
> WhiteNoiz3
With the old model (and I suspect this one too) it's
trained to generate from a single 'seed' pixel in the
center of the image. If you erase the center of the
image, that's when it completely collapses.
|
jekude
The abstract implies that strictly local updates are a
hinderance to high res, however i would have thought there
would be an interesting way to get speed up gains from
neighbor-only traffic on GPUs CAM-style. am i making that
up?
|
> esychology
I think performance is not the only issue for scaling
to larger grids. CUDA Convolution implementation
already utilizes coalescing to improve performance.
The main bottleneck is that in larger grids, cells are
further apart, and it takes more steps for them to be
able to communicate.
|
hidelooktropic
For the unfamiliar, could someone explain what I'm looking
at? The abstract was a little too concrete (heh) for me to
follow.
|
> esychology
If you're familiar with CAs (e.g. Conway's Game of
Life), you can think of a NeuralCA as a CA where the
update rule is given by a neural network. Here we
optimize the neural net weights so that it behaves a
certain way (e.g. grow a lizard from a single seed).
|
> > flir
What are the inputs to the NN? The whole grid, or
just nearby cells? What happens if two NNs overlap
on the same grid? (Gonna go read the paper).
|
> > > esychology
The input to the NN is just the 3x3
neighborhood around a cell.
We can overlap two NNs on the same grid
(through interpolation). Checkout
https://meshnca.github.io
to see the effect. When the brush is in graft
mode, it basically allows you to paint some
regions that will follow a different NN rule.
|
> > > > flir
> The input to the NN is just the 3x3
neighborhood around a cell.Well that
sounds like black magic. Nice. Thanks for
the reply.
|
embedding-shape
Really interesting demo, nicely done :) Would be fun if
switching the "Target Image" when using the second brush
mode in the Growing Demo didn't erase/reset the existing
canvas, so we could "stamp" new things on top of other
images. Small thing perhaps but I got sad when it
disappeared when I wanted to merge a kitten on top of the
chameleon but couldn't :(
|
> bfmalky
You can, just enable the 'transition' switch.
|
WithinReason
You can make the centipede grow longer, which makes sense
given how this works. Or grow a 2nd centipede for extra
points.
|
> esychology
haha yes, also the same with the worm
|
amelius
Why are the images always generated in the same
orientation (upright)? Do the cells have awareness of what
is "up"?
|
> esychology
yeah normally NCAs have a sense of up and left. There
are some isotropic variants that make the perception
fully rotation-invariant.
|
mirekrusin
So the goal is to evaporate it with minimum number of
shots?
|